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Preface 


This text is intended to provide the user of instrumentation with an understanding of the factors that influence 
instrument selection, instrument application, the interpretation of results, and the subsequent planning of 
future operations. In selecting the tools and techniques of measurement, consideration must be given to 
factors like their reliability, accuracy, speed, cost, and role in contributing to the ultimate goal of the 
measurement operation. Because the relative importance of these factors varies with each application, the 
balance among them requires a human judgment. A mathematical formula may be used to describe this 
judgment, but only after the judgment has been made. 

The text also seeks to provide that intuitive understanding of the capabilities and limitations of measurement 
methods that will permit avoidance of an excessive precision and detail beyond that needed by a practical 
engineering problem. Such matching of effort to need not only saves time and labor, but also attests to 
the soundness and maturity of the judgment that has been exercised. 

The chain of variables, components, or steps that comprise a measurement operation can proceed from 
the physical variable to be measured, to a sensor and its installation, to transmission of the signal from 
the sensor to a signal transducer, conditioner, or modifier, to an indicator or recorder, to data analysis 
or manipulation, and to the interpretation and understanding of the data. This text, being directed to the 
instrument user rather than to the instrument designer, treats, in categorical terms, only those items in the 
chain that principally control the accuracy, reliability, and utility of the information acquired; these items 
are the first two or three items of the chain, and the very last item. The quality, sophistication, and pace 
of development of modern instruments for data acquisition and manipulation are such that the intermediate 
items of the chain rarely limit the ability to acquire and to understand the information that is desired about 
the physical variable being studied. 

This material is the substance of a set of thirty 2-hour lectures presented to members of the senior engineering 
staff of what is currently known as the NASA Lewis Research Center. The subjects selected for treatment 
in this text are those which have been found to be most significant in the practical application of instruments 
and of measurement techniques by this staff. The more basic principles are presented in the main text of 
each chapter; additional details appear as appended notes following each chapter. Illustrative examples are 
chosen, as far as possible, to represent practical applications, sometimes in several engineering disciplines. 
Many of these examples are in a form that makes them suitable for use as problems in a formal course 
of instruction. 

An understanding of the language of the differential calculus is necessary; but facility in its operations 
usually is not. To facilitate cross reference, and the text’s use as a handbook, figures and specialized tables 
bear the same number as the section in which they appear. 
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CHAPTER 1. DIMENSIONS 


1 . 0 . Physical quantities . Symbols. The development of physics has included the identification of measurable 
physical quantities representing properties or characteristics of matter, and the establishment of relationships 
among these quantities. Each newly identified quantity has been given a name that is usually widely accepted. 
[Note N1 .0] For convenience, each quantity is also assigned a symbol, although this assignment is arbitrary 
and may vary with the discipline in which the quantity is used, may vary from one text to another, and 
may even vary among different sections of the same text. 

The subject of dimensions deals with the names of these quantities. Each physical quantity, property, 
or attribute may be assigned a dimension characterized by its name, which is descriptive of its nature or 
behavior. For example, mass, energy, and temperature may be said to have the dimensions of mass, energy, 
and temperature, respectively. As a separate matter, the term “unit” characterizes the quantitative magnitude 
of some physical quantity, property, or attribute. For example, the gram, slug, and pound are all units 
of mass; the joule, erg, and British thermal unit are all units of energy; the kelvin, degree Celsius, and 
degree Fahrenheit are all units of temperature. 

The ratio between two quantities having the same dimensions is said to have the dimension of unity. The 
ratio itself is said to be dimensionless or nondimensionaL All numerics (pure numbers like \f2 or ir) 
and transcendental functions (like the logarithm or the sine) are dimensionless. 

The concept of dimensions was introduced by Fourier in 1822. (Ref. 1-1) It facilitates the qualitative 
development and understanding of the laws of physics even before they are converted to quantitative 
relationships through choice of the units of measurement. 

If each physical quantity is considered to have a dimension, it is found that the dimensions of any physical 
quantity can be described as a combination of some or all members of a group of five independent dimensions. 
The selection of which five quantities are to be used in the group is arbitrary, subject only to the condition 
that each be independent of the others (i.e., that none shall be merely a power or product of the others) 
and is generally made on the basis of convenience. The term “independent dimension” is analogous to 
the term “independent variable” in mathematics. 

In this chapter, square brackets around the symbol for a quantity are used to signify the dimensions of 
that quantity. Thus, [A] is read “the dimensions of A.” The dimension itself is represented by an unbracketed 
symbol for that quantity; in this text, that symbol will always be capitalized. The symbol for the quantity 
itself may or may not be capitalized. Thus, if area a and length / are two physical quantities, and A and 
L represent the dimensions area and length , respectively, 

[a] = A = L 2 ; [/] = A 1/2 = L; [a /l] = L 

1.1 Relationships among physical quantities . In most cases, the basic laws of physics are statements 
of proportionalities between physical quantities or powers of those quantities. [Note Nl.l] The conversion 
of proportionalities into equalities requires the assignment of a factor of proportionality. This factor may 
itself have dimensions or, if it is dimensionless, may be a pure number (a numeric) or the ratio of two 
physical quantities that have the same dimensions; the factor also depends on the units in which the physical 
quantities are measured. The subject of units will be treated in the next chapter. 
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Tables 1-1 to 1-5 1 list some quantities of physics and their dimensional representations in a useful 
grouping of independent dimensions. Some tables are arranged to show the parallelism between analogous 
quantities. More complete tables are available in handbooks of physics and engineering. In the column which 
lists definitions of symbols by a formula, the required proportionality sign has been replaced by a 
proportionality factor (often unity) in a manner that provides a fully consistent set of equations for those 
systems of units that have had wide acceptance since 1890. The selection of the value of the proportionality 
factor will be treated in the next chapter , but it is important to note that the dimensions of a physical quantity 
do not depend on its magnitude and are independent of the units in which it is measured. 

Example 1 . 1 . Names, symbols, and proportionalities are illustrated by the following relationships 
among physical quantities: 

(l.l-ls) (force f) oc (mass Af) x (acceleration a) Newton's law 

(1.1-lb) (electrostatic flux *) oc (electric charge Q) Gauss’s law 

(1.1-lc) (magnetic flux $) oc (magnetic pole strength m*) Gauss’s law 

(1. 1-ld) (magnetomotive force T) oc (current /) X (turns n) solenoidal law 

(1.1-le) (force F) oc (current I) x (induction B) x (length L) 

(1.1-lf) (energy change AH') oc (heat capacity C) x (temperature change A0) 

All of the above relationships happen to require a dimensionless factor of proportionality. When 
such a relationship among physical quantities is converted into the corresponding relationship among 
their dimensions, the proportionality sign becomes an equality sign. Thus, 

(1 • 1— 2a) [F] = [M][d) 

(1-1 —2b) [♦] = [Q] 

(1.1 -2c) [$] = [m*] 

(1 - 1— 2d) [?] = [/][«] = [/] 

(1 - 1— 2e) [F] = (/] [fi] [L] 

(1.1-20 [W) = [C][d] 

The following relationships require a dimensional factor of proportionality. The corresponding 
relationship among dimensions may not be written as an equality; it serves merely to define the 
dimensions of the factor of proportionality. 

(l.l-3g) (radiant flux $,) oc (temperature 0) 4 x (area A) Stefan’s law 

(l.l-3h) (energy change AH'') oc (frequency change A»>) Planck’s law 


For ease of later reference, Tables 1-1 through 1-5 appear at the end of this text on pages 208 to 213. 
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(1-1 — 3i) 

(pressure p) <x (density p) x (temperature 0) 

Gas law 

(l.l-3j) 

(mass electrodeposited m) oc (total charge Q) 

Faraday’s law 

Thus, the 

respective factors of proportionality have the following dimensions: 

(1. 1-4g) 

[$,]l[d*A] = W/(L : T0 4 ) 

(Stefan-Boltzmann constant, a) 

(1.1 — 4h) 

= WT 

(Planck’s constant, h) 

(1-1 — 4i) 

\pV\pB} = W/(M0) 

(specific gas constant, R) 

(].l-4j) 

[«]/[£?] = IRT 2 /L 2 

(Faraday constant, F) 


1.2. Psychophysical quantities. These are quantities that represent the perception of physical quantities 
by a human observer, often called the standard observer. 

In illumination engineering, psychophysical quantities are based on the spectral sensitivity of the human 
eye. The word ‘luminous” is prefixed to the the name of the quantity to denote the psychophysical quantity, 
in contrast to the prefix “radiant” for the purely physical quantity. ( Example : luminous flux versus radiant 
flux. A source emitting considerable flux in the deep ultraviolet emits no luminous flux because the radiation 
is invisible.) 

In audiometry, psychophysical quantities are based on the auditory sensitivity of the human ear or on 
human psychological characteristics. ( Examples : loudness, noisiness.) The commonly used psychophysical 
quantities happen to be dimensionless because they are functions of the ratios of two similar physical quantities, 
like pressure or power. 

1.3. The Pi theorem. Dimensionless numbers. If n independent physical quantities enter into some physical 
relation, and the dimensional expressions for these quantities involve a total of k independent fundamental 
dimensions (note that k < 5), the relation among the physical quantities can be expressed as a function of 
n-k independent dimensionless products of these n quantities in the form 

(i.3-i) /(n„ n 2 , n 3 , . . n n . k ) = o 

where 11/ is a dimensionless product of a number of the n physical quantities. The word “product” includes 
the operation of raising any one quantity to some power. This is currently called the Pi theorem. 2 

The scaling of models involves changing the magnitudes of the quantities that enter into any II, without 
changing the magnitude of 11/ itself. Similarly, calibration of an instrument needs to be performed only 
over a range of values of each 11,, rather than over a range of values of each quantity that enters into any 
11,; calibration effort is, thereby, reduced. 

In any one discipline, certain dimensionless products recur frequently. They are then termed dimensionless 
numbers and are often named after a scientist prominent in that discipline. The General Conference on Weights 
and Measures (CGPM) recommends that they be written as N&, where the subscript identifies the scientist. 
Where the meaning is unambiguous, they have also been written as Xx rather than as subscripted numbers. 

( Examples : Reynolds number, N Re or Re, Prandtl number, N Pr or Pr). 

Table 1-6 3 contains a listing of a few dimensionless numbers. More extensive compilations have been 
published (Refs. 1-4 and 1-5). 


The theorem was stated by Riabouchinsky in 191 1 (Ref. 1-2) for the case k = 3, in application to problems in mechanics. It was 
recognized independently by Buckingham and stated for the more general case k > 3, in 1914 (Ref. 1-3). 

For ease of later reference, Table 1-6 appears at the end of this text on page 214. 



1.4. The Pi-zero theorem . The Pi theorem leads to the corollary that a physical quantity y can be expressed 
as the product 7r 0 of other physical quantities (the dimensions of x 0 being the same as the dimensions of 
y) multiplied by a function of dimensionless products of the quantities entering into the physical relationship. 
For example, suppose that y occurs to the m th power in the product IIj of Eq. (1.3-1). Then we take 
x 0 = y Ilf 1/m and rewrite Eq. (1.3-1) in the form 

(1.4-1) y = ir 0 F(U 2t n 3 , . . U n . k ) 

This relation leads to a fundamental law of measurement: The percentage accuracy of measuring y can be 
no better than the percentage accuracy of measuring x 0 . (See Sec. 3.28.) 

A restatement of this law is that to obtain a given accuracy in the measurement of a quantity y, it is necessary 
(but not always sufficient) to compare it with another quantity x 0 that is known to at least the same 
percentage accuracy, x 0 being a product of variables such that the dimensions of x 0 are the same as the 
dimensions of y. 

A particularly simple case, very common in instrument calibration, is n-k = 1, so that there is only one 
product II,. Then the unknown quantity y can be expressed as a simple product of powers of the other 
variables, and the accuracy of knowledge of the unknown is equal to the accuracy of the product, if the 
calibration or comparison procedure itself introduces negligible error. 

In the simplest case, x 0 may be a single physical quantity having the same dimensions as y. 

Example 1.4 a. “Potentiometer of the first kind.” (Fig. 1.4(a)) To measure an unknown 
electromotive force (emf) e xt the current i through a resistor r s is first adjusted until the voltage 
drop across r s is equal to a known standard emf e s , derived from a standard cell or its equivalent 
(e.g., a Zener-diode-based power supply), as indicated by a null detector. Then e s is replaced by 
e x and r x is adjusted until the voltage drop across r x is equal to e x . Then e x = e s (r x /r s ). The value 
of / need not be known but must have remained constant during the entire operation. Here x 0 = e s . 
The percentage accuracy of knowledge of e x is no better than the percentage accuracy of knowledge 
of e s . The ratio r x /r s must be known with negligible error, but the absolute value of the resistances 
need not be known. 

Example 1.4 b. “Potentiometer of the second kind.” (Fig. 1.4(b)) The current / through a fixed 
resistor r 5 is adjusted until the voltage drop across r s is equal to an unknown emf e x , as indicated 
by a null detector. The current i is measured. Here x 0 = ir s . The percentage accuracy of knowledge 
of e x is no better than the percentage accuracy of knowledge of the product ir s . 

This arrangement lends itself to the use of a feedback amplifier to replace a manual operator and 
to the use of an oscillograph to replace the ammeter, in order to achieve rapid emf measurement 
while almost maintaining the null potentiometer feature. 


Null 
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Figure 1 .4 - Null-potentiometer circuits. 



Example 1.4 c. The value r x of a resistor is measured, using a Wheatstone bridge with ratio arms 
r A and r 5 , by comparing it with a standard resistor r s . Here x 0 = r s . The percentage accuracy of 
knowledge of r x can be no better than the percentage accuracy of knowledge of r s and of the ratio 

r A fr B - 


Example \M. The value r of a resistor is measured by determining the voltage drop e across 
the resistor when a measured current / passes through it. Here, 7r 0 = e/i. The percentage accuracy 
of knowledge of r can be no better than the percentage accuracy of knowledge of the ratio of voltage 
drop to current. 

Example YAe . The period T of oscillation of a pendulum of length L swinging in a plane with 
moderate amplitude to a maximum angle a under a gravitational acceleration g is such that 

(1.4-2) T oc {L/g) m \\ + (l/4)sin 2 (a/2)] 

Here, 7r 0 = {L/g) m . The remainder of the expression is a small correction factor whose application 
is usually warranted, although the uncertainty in this factor usually contributes negligibly to the 
inaccuracy of measuring T. 

Example 1.4/. The power loss P from a section Ax of a substantially infinite wire of diameter 
d when it is transverse to a gas stream of velocity v, density p, thermal capacity c, and thermal 
conductivity k is given by 

(1.4-3) Pock* Ax* Ad j^l + (27 r\pcd/k) ]f2 ^ 

when it is heated so that a temperature difference A 8 exists between wire and gas. Here, 
7T 0 — k* Ax • AS and n 2 = vpcd/k. (n 2 is the Peclet number.) At low velocities, n 2 contributes 
moderately to the ratio Pf (k * Ax • A0), and the uncertainty in knowledge of n 2 contributes 
negligibly to the ratio. 4 

Example 1.4g. The mass flow rate m of the gas through a restriction of area A is determined by 
measuring total temperature T t , static pressure p s , and the difference Ap between total pressure p, 
and static pressure. The applicable relationship is m oc C A b [2p s * Ap/ (RT t )} V2 where C, the 
dimensionless discharge coefficient, is principally a function of restriction shape (and secondarily 
of the Reynolds number), R is the specific gas constant, and b is a dimensionless function (whose 
magnitude differs less than 10 percent from unity) of the specific heat ratio of the gas and of p s /p t . 
Here 

(1 .4-4) v 0 = A [: 2p s • A p/(RT,)] 1/2 

and II 2 = C, n 3 = b. The inaccuracy of measuring m is determined principally by the inaccuracy 
of measuring t 0 . A slight contribution to inaccuracy originates from the usual uncertainty in n 2 , 
and a negligible contribution originates from the usual uncertainty in n 3 . 

An Operational Note . The Pi-zero theorem may impose severe limitations on the ability to measure the 
absolute value of a physical quantity. However, it does not inhibit the ability to measure the difference 
or the ratio between the unknown quantity y and a reference quantity tt 0 which, though not adequately 
known in absolute magnitude, is known to be very stable. Thus, the null detectors used in Examples 1.4a 
to 1 .4c may be able to detect emf or resistance differences smaller than the uncertainty in knowledge of 
the absolute value of 7r 0 . 


the thermal -conductivity-type gas analyzer, the conductivity k is deduced from accurate measurements of P and A 6. The velocity 
v must be nonzero in order that changes in gas content may be followed. 
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Resistance-strain-gage bridges, as used for strain, load, or pressure measurement, routinely measure 
fractional resistance changes A R/R so small that the uncertainty bR in knowledge of the absolute value of 
R may be 1000 times A/?. 

1.5. Dimensional analysis. If a physical situation is represented by an equation, and if each quantity 
in the equation is replaced by its dimensional equivalent, the resulting dimensional equation must be satisfied. 
(In the process of substituting dimensions, a pure numeric is replaced by unity.) In a strict mathematical 
sense, this does not imply that each term of a sum must have the same dimensions. ( Example : if F = ma 
and A = 7 rr 2 , then F 4- A = ma + itr 2 and [F] 4- [A] = [ma] 4- [7rr 2 ].) However, in all practical situations, 
one does seek to sum forces, powers, energies, areas, voltages, etc. In such cases, a check on the dimensions 
of each term may be the first step in confirming the correctness of the summation. 

If the physical relation is not known, but the identities of the physical quantities involved are believed 
to be known, it is sometimes possible to determine how the physical quantities should be multiplied together 

in order to describe the physical situation. Suppose that there are n physical quantities y h v 2 y n that 

are known to affect the phenomenon being studied, and that all n quantities can be represented by a total 
of k dimensions (k< 5). To find one of the n-k products 11/ that enter into Eq. (1.3-1). assign an unknown 
exponent a t b, ... to each of the respective y, and seek a product yjV* . . . such that, when each y, is 
replaced by its dimensional equivalent, the exponents of each dimension in the product will vanish, since 
II, is dimensionless. 

Similarly, if a quantity y, is believed to depend on other quantities y 2 , V 3 , . . . one may be led to an 
appropriate tt 0 by replacing each y f with its dimensional equivalent [yj and finding the exponents a , b, 
c, . . . that will satisfy the relation 

(1.5-1) [ yJ-WW*... . 

Then 

(1.5-2) To = yf V* • • ■ 

Operations of the type described in the two preceding paragraphs are termed dimensional analysis. 
Dimensional analysis 

(a) will not determine the functional relationship between the n-k products of Eqs. (1.3-1) or ( 1 . 4 - 1 ), 

(b) will not establish the value of any numerics that should multiply the respective products or be added 
to them, and 

(c) will not assure that the products so determined have any real significance in the physical situation 
being studied. 

On the other hand, the results obtained by dimensional analysis may provide a guide to the conduct of 
experiments to verify that the products do have physical significance, thereby to establish numerical constants, 
and to establish the functional relationships among the various products. (If no product can be formed, the 
indication is that some physical variable has been overlooked.) Consequently, dimensional analysis is a 
tool whose effectiveness in solving any problem is determined a priori by the quality of physical intuition 
available and a posteriori by the amount of experimental evidence that is available. 

Example 1.5a. The impact force Fon a plate normal to an airstream is believed to be dependent 
on the plate area A , the air density p, and the air velocity v. Using the M,L,T system of dimensions, 
we have n = 4 and k = 3. The product F°A b p c \ d is converted into the dimensional form 
(MLT~ 2 ) a (L 2 )* (ML~ 3 ) C (LT“V. Setting the exponents of M, L, and T equal to zero leads to the 
three simultaneous equations a 4- c = 0, a + 2b-3c+d = 0, -la -d = 0. Arbitrarily setting a = 1 
yields b — — 1, c = — 1 , d — —2. Hence, f(pA\ 2 /F) =0, or F = pA\ 2 x (a numeric). 

Example 1 .5b. The period T of a pendulum is believed to be dependent on its length /, the mass 
m at its free end, and the acceleration of gravity g. Using the M,L,T system of dimensions, we 
have n=4, k = 3. The product T a l b m c g d is converted into the dimensional form 
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(T) a (L) fe (M) c (LT -2 )^. Setting the exponents of M, L, and T equal to zero leads to the three 
simultaneous equations a -2d = 0 t b + d = 0, c = 0. Arbitrarily setting a = 2 yields b = - 1 and 
d = 1. Hence, f(T 2 g/L) = 0 or T 2 — (l/g) x (a numeric). 

Example 1.5c. The resonant frequency/ of an electrical circuit is believed to be dependent on 
the lumped values of inductance L, capacitance C, and resistance R of the circuit. All four of these 
quantities may be represented in terms of two dimensions: R and T. Thus, n = 4 and k = 2, so 
that two dimensionless products are to be sought. Converting f a L b C c R d into its equivalent 
dimensional form yields fewer equations than unknowns. However, assuming the expression f a L b C c 
readily yields the dimensionless quantity f 2 LC, and assuming the expression L a C b R c readily yields 
the dimensionless quantity R 2 C/L. Thus, 

F(f 2 LC , R 2 C/L) = 0 

These two groupings offer several advantages: First, since /appears in only one of them, an explicit 
expression for/ will be possible; second, since R appears in only one of them, the expression containing 
R will exclusively represent the effect of energy dissipation. Thus, one may write 

/= (LC)~ xn <t>(R 2 C/L) 

where the function <j> is a numerical multiplier of (LC) -1/2 , which is also the 7 r 0 of Sec. 1.4. 

Example 1.5d. The power loss P from an electrically heated length Ajc of a very thin, very long 
wire of diameter d< when it is transverse to a gas stream of velocity v, density p, specific heat c, 
thermal conductivity k , and viscosity r?, and where a temperature difference A0 exists between wire 
and gas, is to be examined for a likely functional relationship. In the dimensional system H,L,T,0, 
one finds n = 9 and k = 4, so that up to five dimensionless products are possible. It is reasonable 
to expect P to be proportional to Ax and to AO. Since [P/(Ax • A0] = HL _1 T _l 0 _1 , which is also 
[k], we choose P/(k • Ax • A0) as II (It is termed the Nusselt number, N Nu .) If P , Ax, and AO 
are eliminated as being unlikely to appear elsewhere, n becomes 6, so that two more dimensionless 
products are to be sought. We choose the Reynolds number N Re = d\p/ 17 as one of these because 
it is known to enter commonly into heat transfer phenomena in a flowing gas. For the other product, 
we solve 


[p a c b k c ri d ] =(ML' 3 )‘ 1 (HM“ I e _l ) fc (HL _, T _1 e _1 ) f (ML _1 T" 1 )‘ , = 1 


which results in a = 0, b = — 1 , c = 1 , and d = - 1 , so that cr\fk is II3. (It is termed the Prandtl 
number, N Fr ). Hence, 


(1.5-3) 


/ 


\k • Ax • Ad 


d\p ct)\ 

T’ *7 


= 0. 


The first product represents the effect of the input electric power, the second represents the effect 
of gas stream motion, and the third represents the effect of intrinsic gas properties. Equation (1.5-3), 
when rewritten in the form 


(1.5-4) P = k* Ax* AO F(N Re , N Pr ) 

may be compared with Eq. (1.4-3) by noting that the Peclet number is N Pe = N Re N Pr (Footnote 5). 


^This example is the basis of hot-wire anemometry. In its usual applications, the principal instrumental measurement is that of 
the ratio P/(Ax • A6)\ any deviation of k from its nominal value is treated as a systematic correction. The principal independent variable 
is the Reynolds number; the Prandtl number may usually be assumed to remain constant. 
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1.6. Empirical coefficients . The unknown numerics that appear in the equations derived by dimensional 
analysis can sometimes be determined by further theoretical analysis; at other times, recourse must be had 
to purely empirical determinations; and at still other times, the value predicted by theory must be multiplied 
by an empirically determined coefficient as the result of experiment. Much of the work in the calibration 
of instruments consists of the determination of such empirical coefficients, often termed calibration factors 
A complete calibration includes the determination of the value of the calibration factor over the full range 
of II, values that will be encountered during the use of the instrument. 

Example 1.6 a. In Example 1.5a, Bernoulli’s principle will indicate that the numeric should be 
1/2, but experiment will show that this numeric must be multiplied by an empirical coefficient C D 
(the drag coefficient) that is principally a function of the Reynolds number N Re and of the shape 
of the frontal surface of the plate. 

Example \M. In Example 1.5b, theoretical analysis can usually provide the numerics that will 
correct for the fact that any real pendulum is a compound one rather than a simple one. However, 
empirical determination is required of the correction for the ambient atmosphere, which contributes 
entrained mass and viscous damping. The Stokes number N ^ may be the appropriate independent 
variable. A similar empirical correction also applies to tuning forks and to piezoelectric crystals 
vibrating in the air. 

Example 1.6c. In Example 1.5d, experience shows that the form of F(N Re , N Pr ) is 
F = a + bN Re N Pr and that only empirical determination of the four constants a y b>c,d in the actual 
operating range of N Re and N Pr can yield the value of F to an acceptable degree of accuracy. 

NOTES FOR CHAPTER 1 

Nl.O Occasionally, the widely accepted name may not be semantically correct. For example, molecular 
weight means relative molar mass or “multiple of unit atomic mass”. (Since 1960, by international agreement 
among chemists and physicists, unit atomic mass has been 1/12 of the mass of one atom of carbon 12.) 

Nl.l For example, 

(a) the acceleration of a given mass is proportional to the force acting upon it (Newton’s law); 

(b) the flux emanating from a point source (of electric charge, magnetic charge, or radiation) is proportional 
to the strength of the source (Gauss’s law); 

(c) the magnetomotive force generated by a current passing through a coil is proportional to the magnitude 
of the current and to the number of turns in the coil (solenoidal law); 

(d) the force on a current-carrying conductor in a magnetic field is proportional to the magnitude of the 
current and to the density of the magnetic field (a consequence of Ampere’s law); 

(e) the energy required to raise the temperature of a given mass of a passive solid is proportional to the 
specific heat capacity of the material and to the magnitude of the temperature change. 

(f) the force between two electric charges is inversely proportional to the square of the distance between 
them (Coulomb’s law). 
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CHAPTER 2. UNITS 


2.0 Terminology. In this text, “In” will represent the Napierian (natural) logarithm, and “log” will 
represent the common logarithm. 

In this chapter, the following symbols will be used to represent the numerics (nondimensional numbers) 
indicated: [Note N2.0] 

c a = 0.980665 

c m = 0.45359237 

Cq = 2.99792458x10 10 

c p = 1.01325 

The use of these and of the numeric 0.3048 (= 12 x 0.0254) will permit statement of the exact ratios 
between units of the same physical quantity. 

2. 1 Coherent systems of units . As indicated in Chapter 1 , the relations among physical quantities are 
usually described by proportionalities. Quantitative measurement of these quantities requires that these 
proportionalities be converted into equations. This conversion requires statements of the units in which each 
physical quantity must be measured in order to make the equation valid. Modern publications usually make 
these statements explicitly. However, earlier publications sometimes tacitly assumed the reader’s knowledge 
of the system of units used by the author. In order to allow full understanding of such publications, which 
are still authoritative and often unique, a brief review will be given of those systems of units that have 
had wide acceptance. More complete treatments exist (Refs. 2-1 to 2-3). 

The systems of units were constructed by 

(a) arbitrarily selecting a unit value for five fundamental physical quantities (four of these have invariably 
been mass, length, time, and temperature), and 

(b) deciding on the form of the equations to be used to define all the remaining quantities in terms of 
the fundamental ones. This second step is tantamount to selecting the constants of proportionality that will 
convert relations like (1.1-1) into equations. In fact, the use of the defining equations in Tables 1-1 to 1-5 
and the selection of the constant /3 that appears in Table 1-3 fulfill the requirements of this step. A system 
of units constructed in this manner has been termed “a coherent system of units.” 

Sections 2.2 to 2.7 provide brief descriptions of the systems of units in common use since the creation 
of the International Committee on Weights and Measures (CIPM) in 1889. Table 2-1 1 lists, for each of 
several physical quantities, magnitudes in these systems of units that are equal to each other. Table 2-2 1 
lists some relationships among the units that are useful in converting from one system to another and in 
the preparation and use of equations describing physical phenomena. 


i 


For ease of later reference, Tables 2-1 through 2-3 appear at the end of this text on pages 215 to 219. 



2.2 Traditional systems of mechanical units. Historically, the fundamental mechanical units developed 
and used in the late nineteenth century on the European mainland were the centimeter (cm), gram (g), and 
second (s). The system of units based on these was termed the cgs system. The units of force and energy 
were the dyne and the erg, respectively. 

In English-speaking countries, two systems became common. In one, the fundamental units for length, 
mass, and time were the foot (ft), pound (lbm), and second (s); the system of units was termed the fps 
system, and the unit of force was the poundal (pdl). In the other system, the fundamental units for length, 
mass, and time were the foot, slug, and second; the system of units was termed the fps gravitational system 
( fpsg system), and the unit of force was the pound force (lbf). Newton’s law in the two English systems was 

(2.2-1) 1 pdl = 1 lbm x 1 ft/s 2 

(2.2-2) 1 lbf = 1 slug x 1 ft/s 2 

The relation among units for the same physical quantity is 

( 2.2-3) = 0.03048/c a = 1/32.17405 

1 lbf 1 slug 

In emulation of the dual use of the word “pound,” a unit of force termed the gram force (gf) also became 
common; thereupon, the unit of mass became the gram mass (gm). The relations among the units of mass 
and force are 

(2.2-4) 1 lbf = g n x 1 lbm 

(2.2-5) 1 gf = x 1 gm 

where g n is the standard acceleration of gravity (980.665 cm/s 2 and * 32.17405 ft/s 2 ). In publications where 
the unit of force is the pound force (or gram force) and the unit of mass is the pound mass (or gram mass), 
the factor g n appears explicity in some equations, often with just the symbol g. 

2.3 Traditional systems of thermal units . The Celsius or centigrade degree (°C) has been associated with 
the cgs system of mechanical units. Absolute temperature is then measured in kelvin (K) and energy in 
gram-calories (cal or gcal). 

[In 1967, the CGPM elected to replace “degree kelvin” with “kelvin” (K)]. 

The Fahrenheit degree (°F) has been associated with the fps and fpsg systems. Absolute temperature is 
then measured in degrees Rankine (°R) and energy in British thermal units (Btu). 

The relations among quantities in each system are 

(2.3-1) K = °C + 0 O 

(2.3-2) °R = °F + 0 O — 32 

0 O = temperature (in K or °R) of the triple point of H 2 0 

(2.3-3) 1 gcal = energy to raise 1 gm of H 2 0 from 15 to 16 °C 

(2.3-4) 1 Btu = energy to raise 1 lbm of H 2 0 from 39 to 40 *F 

2.4 Traditional systems of electric and magnetic units . In the field of electrostatics, the fifth fundamental 
unit has often been taken as the permittivity of a vacuum; the other four units are those of the cgs system. 
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The complete system is termed the cgs electrostatic system of units ( esu system). Names of electrical quantities 
often bear the prefix “stat” (e.g., statampere, statfarad, stathenry, statvolt). 

In this system, e 0 = 1 statfarad/cm, and fi 0 — (1/cq) stathenry/cm. 

In the field of magnetics, the fifth fundamental unit has often been taken as the permeability of a vacuum; 
the other four units are those of the cgs system. The complete system is termed the cgs electromagnetic 
system of units (emu system). Names of quantities often bear the prefix “ab” (e.g., abampere, abfarad, 
abhenry, abvolt). In this system, fi 0 = 1 abhenry/cm and e 0 = (1/cq) abfarad/cm. 

Some writers, like Gauss, have used e 0 = 1 statfarad/cm when dealing with electrostatic phenomena and 
/xq = 1 abhenry/cm when dealing with magnetic phenomena. Consequently, some physical quantities are 
measured in esu, others in emu. The system is called the cgs Gaussian system. 

2.5 Rationalization . The esu, emu, and Gaussian systems all use 0 = 4x in the equations listed in Table 
1-3. This factor enters principally because the surface of a sphere subtends 4 tt steradians at the sphere’s 
center. Some writers, like Heaviside and Lorentz, have set 0 equal to unity, thereby altering the relative 
magnitudes of some electrical and magnetic units, but reducing the frequency of occurrence of the numeric 
4 tt. The use of 0 = 1 was termed “rationalization”; thereby, systems using 0 = 4i r were termed 
“unrationalized”. Heaviside and Lorentz also used the definitions of e 0 and /x 0 that Gauss had used; the 
resultant system of units is termed the cgs Heaviside-Lorentz (HL) system. 

2.6 Later systems of units. The “absolute” system of electric and magnetic units (which includes the 
ohm, ampere, volt, coulomb, farad, henry, watt, and joule) consists of simple decimal multiples of the 
emu that are of practically useful magnitudes. For about half a century prior to 1948, a set of “International” 
electric and magnetic units was in almost universal use. These differed very slightly from the absolute units 
because they were based on practically realizable international standards of the ohm and the ampere; units 
of other electric and magnetic quantities were defined in terms of these two standards. After 1948, 
improvements in technology made possible the practical realization of other absolute-system standards, and 
the older set of International units became obsolete. [Note N2.6] 

In 1901, Giorgi pointed out that a simplification in the formulas for electromechanical phenomena could 
be achieved by a redefinition of the unit of permeability. The resultant unrationalized meter-kilogram-second 
system ( mks system) uses the meter and kilogram as fundamental units, rather than the centimeter and gram, 
and takes fig = 10 ” 7 henry/meter as a fifth fundamental quantity. The system is, in effect, an mks 
electromagnetic system of units. Names of electric and magnetic units are those of the absolute system. 
The units of force and energy are the newton and joule, respectively. 

The rationalized meter-kilogram-second system ( MKS system) is like the mks system except that, since 
0 = 1 instead of 4w in the formulas of Table 1-3, the value of is taken as 4 tt x 10 “ 7 henry /meter. 
Units of force and energy remain the newton and joule, respectively. 

2.7 The SI units. Gradual acceptance of Giorgi’s ideas has led to the adoption of the Systeme International 
(SI )— a rationalized system (Ref. 2-4). It is not to be confused with the “International” system discussed 
in Sec. 2.6. The current fundamental units are the meter, kilogram, second, kelvin, and ampere (rather 
than iiq, as in the MKS system). Additional basic units that have been defined are the mole as the unit of 
quantity and a psychophysical unit of luminous intensity, the candela. These are defined in Table 2-1. 

Table 2-1 lists some common physical quantities, the names of the SI units in which they are measured, 
the abbreviations for these units, and the quantitative equals of these units in other systems of units. 
Reference 2-5 describes these units and other less common ones. 

Table 2-3 lists the prefix symbols that designate multiples of a unit. (Mass is described as a multiple 
of a gram, not of a kilogram.) There is international agreement on the names of the units and on the symbols 
for the units, but, as indicated in Sec. 1.1, symbols for the physical quantity itself must be defined in each text. 

Some dimensional constants are listed in Ref. 2-6. 

2.8 Dimensionless units . Some physical variables are measured in units that are themselves dimensionless, 
usually because they represent the ratios between two quantities that have the same dimensions and that 
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are both measured in the same units. Alternatively, the units may represent some transcendental function 
of such a ratio. In particular, the logarithm of the ratio may be used if the range of values that the physical 
quantity may acquire in any single application is very large. 

Example 2.8 a. Plane angle represents the ratio of the length of a circular arc to its radius. 

Example 2.8 b. Solid angle represents the ratio of the area of a portion of a sphere’s surface to 
the square of the sphere’s radius. 

Example 2.8 c. Electric phase angle may represent the angle whose tangent is the ratio of the reactive 
component of a current to the resistive component of that current. 

Example 2.8 d. Phase angle between two time-varying sinusoidal signals of the same period may 
represent the ratio of the time displacement between the two signals (at some given ordinate, usually 
the mean value) to the period. 

Example 2.8e. The bel (B) is used in the measurement of power P , but the most common unit 
is the decibel (dB). If M is the number of bels and m is the number of decibels, 

(2.8-la) 0. Im = M = log(f7P 0 ) 

where P 0 is some other value of power. Thus, m and M are measures of the ratio between two 
quantities having the dimensions of power. 

If dimensional considerations show that P is proportional to the square of the amplitude of some 
other physical variable X (like voltage, current, displacement, or pressure), then 

(2.8- lb) m = 20 log(X/Xo) 

where Xq is some other value of X. Note that m remains a measure of power ratio. 

Example 2.8/ The neper (Np) is also considered to be a measure of the ratio between two quantities 
that each have the dimensions of power P. It actually is a measure of the square root of the power 
ratio. If m N is the number of nepers, 

(2.8-2a) m N = (1/2) In (P/P 0 ). 

In parallel with Eq. (2. 8- lb), one may write 

(2.8— 2b) m N = ln(X/Xo) 

if P is proportional to X 2 . From Eqs. (2.8-la) and (2.8-2a), 

(2.8-3) 20 m N = m In 10 

Example 2.8 g. If, in Eq. (2.8-1), the quantity P 0 (or Xq) is assigned a fixed, standardized value 
termed a “reference value,” the quantity P (or X) may be described as “m decibels re P 0 (or Xo)” 
or as being “m decibels above or below P Q (or Xq)”, depending on whether P > P 0 or P < P 0 , 
respectively. The quantity m, then, is often said to be the “level” of whatever physical variable 
is being measured. 

In acoustics, where the term “bel” originated, common choices are P 0 = 1 picowatt when P 
is sound power level, and Xq = 200 picobar when X is sound pressure level. In the field of 
electrical transmission, P 0 = 1 milliwatt is sometimes used. 



Example 2.8 A. In psychoacoustics, loudness level , is measured in phons. The loudness level 

of a sound, in phons, is numerically equal to the value of m in Eq. (2.8-lb) when Xq = 200 picobar, 
and X is the acoustic pressure of a 1 kHz tone that is judged equally loud by a standard auditor. 
Empirical curves exist that indicate the relation among m pt X, and either the frequency / of pure 
tones or, alternatively, the center frequency /of relatively narrow bands of white noise (commonly, 
the bandwidth is one third or one ninth of the center frequency). (Ref. 2-7) 

Loudness , L , of a complex sound is measured in sones . If m s is the loudness of a sound in sones, 
then 

(2.8-4) 16 m s = (/?/p 0 ) log 4 

where p 0 is 200 picobar and p is the acoustic pressure of a 1 kHz tone that is judged equally loud 
by a standard auditor. The perceived loudness is approximately proportional to the 0.6 power of 
the acoustic pressure. 

Note that when m p = 40 phon, we have m s = 1 sone. In general, 

(2.8-5) log m s = (0.1 m p - 4) log 2. 

Example 2.8 t. The noise figure , N , of an electronic device has been represented by 
(2.8-6) N (in dB) = 10 log F 

where F is the ratio between the actual noise power at the output of the device and the noise power 
that would have appeared at the output if the device itself had contributed no noise. (But see, also 
Sec. 5-19.) 

Example 2.8/. In audio communication, the volume unit, vu, of a complex audio waveform 
is defined as the numerical value of m in Eq. (2.8- la) when P is the power developed in a 600-ohm 
resistance. Practical VU meters are ac voltmeters that measure the voltage across a 600-ohm load. 
The scales of such VU meters that conform to nationally accepted 1984 standards are marked to 
correspond to P 0 = 2.5 mW in 600 ohms. 

Example 2.8/r. In electrochemistry, pH is defined as 

(2.8-7a) pH= -log 

(2.8— 7b) <2//+ a c 

where a is hydrogen ion activity and is hydrogen ion concentration. When pH is determined 
by measuring the emf e generated in an electrochemical cell, 

(2.8-7c) pH = 5040(e - e 0 )T'^ i kelvin per volt 

where e and e 0 are in volts, e 0 is an instrumental constant that may be dependent on T , and T is 
the temperature of the cell, in kelvin. 

Example 2.8/. The bolometric magnitude , m qt of a stellar object is given by 
(2.8-8) m q = -2.5 log (£,/£*>) 

where EqO = 2.5 X 10 8 W/m 2 and E q is the irradiance, in W/m 2 , as measured by a 
bolometer. 
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The absolute magnitude, M q , of a stellar object is defined as 
(2.8-9) M q — m q — 5 log (/// 0 ) 

where / 0 = 10 parsec and / is the object s distance from the Earth, in parsec. 

The visual magnitude, m x of a stellar object is given by 

(2.8-10) m, = —2.5 log (£ v /£ v0 ) 

where £ v0 = 2.6 x 10 ~ 6 lux and £ v is the illuminance, in lux, as measured by a standard 
observer. 


2.9 Dimensionless numbers . The dimensionless products that are commonly called “dimensionless 
numbers” were described in Sec. 1.3 and illustrated in Table 1-6. In algebraic form, they suffice for the 
purpose of dimensional analysis, as described in Sec. 1.5. However, before such a dimensional number 
can be used quantitatively in a numerical computation, its numerical value must be determined in such manner 
as to be independent of the particular unit in which each component physical quantity is measured; only 
then can diverse workers agree on a common “correct” value in any particular situation. The procedure 
for establishing the correct value of a dimensionless number with the fewest conversions is as follows; 

Step 1. Each symbol representing a physical quantity is replaced by both the magnitude and the 
name of the unit in which that quantity has been measured. 

Step 2. When the dimensionless product is formed, names of units are cancelled or otherwise 
treated as though they were algebraic quantities. 

Step 3. If some names of units remain in the final product, they are caused to be cancelled by 
multiplying the product by unity, where unity is the ratio between appropriately selected equal amounts 
of the same physical quantity, chosen from Table 2-1 or Table 2-2. 


Example 2.9 a. Given N Pr = cy/k, to find N Pr for air when 


c = 0.25 Btu # lbm" •°F~ , rj = 1.7 x 1(T 4 P (P * poise), k = 0.014 Btu»ft *hr # °F . 

Btu . ft # hr #0 F 

Step 1 . N Pr = 0.25 — x 1 .7 x 10 ~ 4 P x 


lbm»°F 


0.014 Btu 


Step 2. 


N Pr = 


0.25 x 1.7 x 10' 
0.014 


P*ft*hr 

lbm 


Step 3. 


Multiply by 


1 gm 1 lbm 

x x 

lP*cm # s 454 gm 


30.48 cm 3600 s 

x ; 

ft hr 


then N Pr = 0.73. 


Example 2.9 b. Given N Re — vdfv , to find N Rv when v = 0.09 cm 2 /s, v = 400 ft/s, d = 0.03 in: 


Step 1. 


a 

N Re = 400- x 0.03 


s 

in X r 

0.09 cm 2 


Step 2. 


s Re = 


400 x 0.03 
009 


ft«in 

x — ~ 

cm" 
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12 in (2.54) 2 cm 2 

Step 3. Multiply by x 

ft in 

then, N Re = 1.03 x 10 4 


Example 2.9 c. Given N Eu = p/(pv 2 ), to find N Eu when p = 1 bar, p = 1 g/cm 3 , v = 3 m/s: The 
combination of steps leads to 


cm 

N Fu = 1 bar x x 


1 g 9 rrr bar # rrf 


10 5 N 1 kg # m 10 3 g 
X r X X X 


NV 


kg 


IQ" 6 m 3 


cm 


100 

~ 9 ~ 


2.10 Coherent equations. The relation among physical quantities is usually written as an equation 
representing the relation among algebraic symbols representing those quantities. Such an equation, rather 
than a proportionality, is necessary in order to establish the quantitative relation among those quantities. 
Such an equation is quantitatively correct if each algebraic symbol is replaced by its value in the same coherent 
system of units. 

However, the equation remains correct, regardless of the unit in which any quantity is measured, if the 
following procedure is used: (1) Each algebraic symbol is replaced by both the magnitude and the name 
of the unit in which that quantity has been measured; (2) names of units are cancelled or otherwise treated 
as though they were algebraic quantities; (3) if the names of the units remaining are not to one’s liking, 
the result is multiplied by unity as often as necessary, where unity is the ratio of appropriately selected 
equal amounts of the same physical quantity, like those listed in Tables 2-1 and 2-2. 

The practice of replacing an algebraic symbol by both the magnitude and the name of the unit of 
measurement is a convenient expedient in those disciplines which have traditionally mixed units from several 
coherent systems. It also obviates the need for converting each unit of measurement into the unit of a single 
coherent system and usually reduces the number of arithmetic operations required. It is also convenient 
in using equations that involve a dimensional factor of proportionality, like those resulting from relation 
(1.1-3). 

Example 2.10a. Use of coherent units. To find the force between two I -coulomb charges one 
meter apart in vacuo: 

Solution 1: In SI units (a rationalized system), F = Q 2 / (lireol 2 ) , where e 0 = 10 n /(47TCo), Q = 1, 
/ = 1. Then F - CqX 10~ n newton. 

Solution 2: In mks units (an unrationalized system), F = Q 2 /(€q1 2 ), where c 0 = 10 n /co* Q = 1, 
/ = 1. Then F = c§ x 10 ~ n newton. 

Solution 3: In cgs electrostatic units (an unrationalized system), F = Q 2 /(t(Jl 2 ), where e 0 = 1. 

Conversions: Q = 1 C x (cq/ 10) statcoulomb/C = (cq/ 10) esu of Q 
/ = 1 m x 100 cm/m = 100 esu of / 


Then 

Co/100 , , 

F = dyne = Co x 10 6 dyne. 

1 x 10 4 

Solution 4: In cgs electromagnetic units (an unrationalized system), F=Q 2 /(t 0 l 2 ) where 
*0 = (1/cg). 

Conversions: Q = 1 C x (0.1 abcoulomb/C) - 0.1 emu of Q 
l = 1 m x 100 cm/m = 100 emu of / 
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Then 


(O.ir 7 

F = dyne = eg x 10 6 dyne. 

(l/cg) x 10 4 7 

Example 2.10 b. Use of mixed units. Given F = ma, m = 3 lb, a = 2 ft/s 2 , to express F in 
newtons: 


F = 



c m kg 0.3048 m 
x x x 

lb ft 


1 N»s 2 
1 kg*m 


= 6 x 0.3048 


x c„ 


newton 


Example 2.10c. Use of mixed units. Given F = ma , m = 3 lbm, a = 2 ft/s 2 , to express F in lbf: 


r / 1L 2 ft\ 1 lbf 6 

F = ( 3 lbm x — ) x = lbf, since g n = 32. 1740 ft/s 2 

V s 2 / g„ lbm 32.2 


Examples 2.9a, b, and c further illustrate the use of mixed units. 

2.11 Numerical equations . The use of coherent equations of the type described in Sec. 2. 10 is necessary 
in the mathematical analysis of physical situations. Their character has been taken for granted in most texts. 
However, where routine repetitive calculations are to be made, it is often more convenient to write an equation 
which is correct only when each quantity is measured in specified units. Quite often, in such cases, there 
is also the tacit assumption that other quantities remain invariant; the symbols for these other quantities 
usually do not even appear in the equation. 

Example 2.11a. Equation (2.8-9) is often written as 
(2.11-la) M q m m q - 5 log / + 5 
where / must be in parsecs, or as 
(2.11- lb) M q = m q + 5 log p + 5 
where p must be the heliocentric parallax in seconds of arc. 

Example 2.11 b. If v = speed of sound in air, a commonly used expression is 
(2.11-2) v = 49.0 \ff 

where air temperature T must be in °R, v must be in ft/s, and the assumptions have been made 
that specific-heat ratio 7 is 1.4 and that relative molar mass is 28.96. 

Example 2.11c. The saturated vapor pressure of many chemical compounds has been written as 
(2. 11 -3a) log p = B - 0.05223 A/T 

where p must be in torr, absolute temperature T must be in kelvin, and A and B are tabulated constants 
for each compound. 
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The corresponding coherent equation is 
(2. 11 -3b) pfp B = exp [- AW/(RqD ] 
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where p B is a constant for each compound, AW is the compound’s molar heat of evaporation, and 
R 0 is the universal gas constant. Thus, 

(2. 11 -3c) A = (AW7/2o)ln 10; B = log p B . 

Equation (2. 1 1 -3b) is valid for any units of pressure and molar heat, provided the procedure of 
Sec. 2.10 is followed. 

Example 2.11 d. The volumetric rate of flow V of a gas through a very thin orifice of area A , 
caused by a difference A p, under molecular-flow conditions (N^ n > > 1, where N Kn = mean free 
path/orifice diameter), can be written as 

(2. 11 -4a) V=3MA(Ap/p) \/(T/m w ) 

where V must be in liter/s, A in cm 2 , and T in kelvin; m w is the relative molar mass, and p is the 
upstream pressure in the same units as A p. 

The corresponding coherent equation is 

(2. 1 1 — 4b) V = A(A p /p) \l[R 0 T/(2zm w )] 

where R$ is the universal gas constant. 

Example 2. lie. If y is the deflection of the end of a spring under the weight of a mass attached 
to the end, if the spring itself is of negligible mass, and if/ is the natural frequency of the mass- 
spring combination, a useful rule of thumb is 

(2. 11 -5a) fy = 10 

if/ is in hertz and y is in inches. This equation also applies to a simple pendulum of length y. 
The coherent equation that leads to Eq. (2.11-5a) is 

(2. 1 1— 5b) f 2 y = g/(4Tr 2 ) 

where g is the local acceleration of gravity. 

This subject is treated more completely in Sec. 8.4. 

2. 12 Conversion of numerical equations. A numerical equation that is correct only in one set of units 
can be converted to a numerical equation that is correct only in another set of units by the following procedure: 

1. Replace each symbol x ( by x/Uj, where u 4 is the name of the unit specified for the 
original equation. 

2. Multiply each * x f /Uj by unity 


1 = AjUj/CBiVj) 

where Vj is the name of the new unit of x,, and A* and are numerics obtained from Table 2-1. 

3. After appropriate combinations of the numerics, the resultant equation will be correct for the 
new units. 
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Example: Given Eq. (2.11-2), to express T in kelvin (K) and v in km/hr: 
Since step 1 leads to 


v(s/ft) =49.0 (77°R) 1/2 , 


we have 


s 1 hr 1000 ft /T 1.8 'R\ 1/2 

ft 3600 s 0.3048 km \°R K J 

so that 

v hr/km = 72.1 V(77K). 

Hence 

v = 72.1 Vr 


if v is in km/hr and T is in kelvin. 


NOTES FOR CHAPTER 2 


N2.0 


c a is the numerical value of the standard acceleration of gravity when it is expressed in 
dekameters per (second) 2 . 

c m is the number of kilograms (mass) in one pound (mass). 

Cq is the numerical value of the speed of light in vacuo when the speed is expressed in 
centimeters per second. 

c p is the number of bars in one standard atmosphere. 

By international agreement, these numerics are exact. 


N2.6. The differences between the International and the absolute units did not exceed 0.05 percent. 
For example, 

1 International ohm = 1.00049 absolute ohm 
1 International ampere = 0.99985 absolute ampere 
1 International volt = 1 .00034 absolute volt 

For a few years after 1947, manufacturers of new precision instruments marked them explicitly to indicate 
that they were in conformity with the absolute system. 

A systematic correction may be required to the marked values of precision resistors or other precision 
components manufactured before 1948 (hence, marked in the earlier International units) if they are to be 
used or compared with components manufactured after 1947, that are marked in absolute units, and if 
discrepancies on the order of 0.01 to 0.05 percent are significant. 
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CHAPTER 3. THE ACCURACY OF MEASUREMENTS 


3.0 Introduction . Symbols . Among the many considerations that influence the choice of a measurement 
device or technique, one that can be described in precise mathematical language is the accuracy of 
measurement. Some other considerations that may be more important, yet less precisely definable, will 
be treated at the end of this chapter, when their relative weight may be better appreciated. 

In making a measurement of a complex physical quantity by use of a number of instruments and operational 
steps, it is generally possible to state what inaccuracy was contributed by each instrument and by each 
operational step, and how each of these inaccuracies contributed to the inaccuracy of the final result. If 
this last value is lower than that required of the final result, the question arises whether appreciable reduction 
in time and in complexity of operations, and a concomitant increase in reliability, could have been achieved 
by using less accurate instruments and techniques; if the final inaccuracy is higher than that required of 
the final result, the problem is to determine at what points it is most feasible to improve accuracy in order 
to attain the goal of the measurement. 

The ultimate accuracy of the final result is determined not so much by the accuracies of the instruments 
used as by the manner in which they are installed, the conditions under which they are used, and the properties 
or behavior of the physical variables that one is trying to measure. Each of these items is one of the independent 
variables upon which the final result depends. The understanding of the way these variables may behave, 
and of how they may contribute to the accuracy of the final measurement, is the principal goal of this chapter. 

The symbols that are used frequently in this chapter are listed here. Infrequently used symbols will be 
defined as they are introduced. 

e error 

E fractional error 

LE limit of error 

n number of measurements 
rms root mean square 

v residual 

w weight 

x independent variable 

v dependent variable 

Ay systematic correction 

6y random error 

Subscripts: 

a arithmetic 

in indicated 

p probable 

tr true 

o root mean square 

0 mean 

Section 3.19 contains some combination symbols that are used frequently in this text. 


Second subscripts: 

0 of the mean 

1 of a single observation 
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3.1 True and indicated values . In most experiments involving measurement of a physical quantity y, 
one is interested in the relation between the true value y\ T of the quantity and the measured or indicated 
value v in . 

Examples are 

(a) The voltage £ lr of a power supply whose output is indicated to be £ in 

(b) The resistance R tr of a resistor whose marked value is /? in 

In some applications, the distinction between y tT and y in may be considered negligible. Examples are 

(c) In kinematic calculations, the effects of latitude and longitude on the acceleration due to gravity 
are often considered negligible, and, for convenience, the standard acceleration g n (Table 2-1) 
is assumed to exist at sea level 

(d) For convenience in making the buoyancy correction in weighing, air is often assumed to be at 
“standard temperature and pressure” (STP), namely, 0 °C and 1 standard atmosphere 

(e) For convenience in optical system design, the speed of light in air is often assumed to be c 0 cm/s 
(Sec. 2.0) 

(f) For convenience in electromagnetic calculations, the permeability and permittivity of air are often 
taken as and e 0 , respectively (Table 2-1) 

In other applications, the distinction between y tT and y in may be one of the principal objects or concerns 
of the experiment. Examples are 

(g) Determining the static temperature of a flowing gas from a thermometer immersed in the gas 

(h) Radiation pyrometry of a warm or incandescent object that is not a black body. 


3.2 Sources of measurement error. Some sources of measurement error are of the following types: 

A. Variation in the physical quantity being measured 

B. Disturbance of the physical variable by the presence of the measuring instrument 

C. Errors introduced by the mode of installation of the instrument 

D. Deficiencies of the instrument itself 

E. Errors contributed by a human observer 

Practical experience has shown that, in many applications, the above listing also is a listing in order of 
decreasing importance. Source A is often overlooked or neglected because it is not apparent or because 
it is difficult to establish quantitatively, especially if repetitive observations cannot be made. Sources B 
and C represent interactions between the instrument and the phenomenon being studied. Source D usually 
receives the greatest attention because a wealth of sophisticated laboratory equipment and techniques exists 
for its study; but this same technological competence also makes it possible for source D to contribute only 
slightly to the overall inaccuracy of the measurement. Source E is usually significant only when one approaches 
the limits of human perception in the measurement operation. 

Examples of operations affected by type A sources are 

(a) Measuring the radiance of the Sun, or the radiance or albedo of the Earth 

(b) Measuring the intensity of radio signals reflected from the ionosphere 

(c) Measuring weak radioactivity in the presence of cosmic or terrestrial radiation 

(d) Measuring magnetic properties of iron alloys of uncertain history 

(e) Measuring a weak electrical signal in the presence of electronic noise 

(f) Measuring electrical conductivity of a chemically impure electrolyte 

(g) Measuring strains in the structure of a bridge carrying traffic 

(h) Measuring sound pressure levels in an outdoor environment 

(i) Measuring the drag of an airplane in turbulent air 

(j) Measuring fluid flow rate at the exit of a pump, turbine, or river 


20 



(k) Deducing bulk velocity of a fluid in a pipe from a linear-velocity measurement along a single 
path, when the velocity distribution across the pipe is not known 

(l) Measuring bulk density or bulk velocity of a fluid when there is two-phase flow' or nonisothermal 
flow' because of the presence of local sources or sinks of heat 

(m) Measuring pressure in a high-vacuum chamber in which there are gas sources or sinks 

(n) Measuring local droplet size in a spray, by light scattering, when there are many droplets in the 
path of observation 

(o) Measuring local temperature or concentration of a hot gas by use of laser beams, when aerosols, 
particulates, or severe gas density gradients are in the optical path 

(n) Radiation pyrometry of a hot gas in which there is chemical activity or chemiluminescence 

(q) Radiation pyrometry of a turbine blade or wheel that reflects radiation from adjacent blades or 
other hot engine components. 

(r) Determining the rate of heat transfer between a fluid stream and an immersed thermometer bulb, 
in the presence of stream turbulence. 

Examples of type B sources are 

(a) A voltmeter draws current from a circuit; an ammeter produces a voltage drop in a circuit. 

(b) A strain gage on thin sheet metal changes the local stiffness. 

(c) A vibration pickup on a small or flexible object changes the mode, amplitude, or frequency of 
the object’s vibration. 

(d) An ionization gauge on a small high-vacuum chamber acts as a pump or as a source to change 
the pressure in the chamber. 

(e) A head-type, vortex-type, or turbine-type fluid flowmeter produces a pressure drop and a flowrate 
change in the pipe carrying the fluid. 

(f) A pitot-static probe or a temperature probe or the support of either affects the fluid flow in a pipe. 

(g) A thermocouple in contact with a small object changes the object’s temperature by conduction 
of heat along the thermocouple leads. 

Examples of type C sources are 

(a) Contact resistances affect the two-terminal measurement of an electrical resistance. 

(b) Capacitance of the cable connecting a piezoelectric pressure transducer to an electrometer affects 
the charge measurement. 

(c) Thermoelectric effects affect low-level dc emf measurements in a circuit with conductors of different 
materials. 

(d) Unclean windows affect radiation pyrometry or optical pyrometry of turbine blades or wheels 
in a jet engine (so does vignetting). 

(e) Dead time of a Geiger-Mueller tube limits the number of ionizing events that can be counted. 

(f) Improperly located current meters produce incorrect averaging of river flow. 

(g) Conduction of heat along the sheath affects the indication of a thermocouple probe measuring 
the temperature of fluid in a pipe or tank. 

(h) Heat loss by radiation and conduction affects the temperature acquired by a thermometer bulb 
in a stream of hot gas. 

(i) Proximity of the wall, or of adjacent probes, or of the support affects the indication of a static- 
pressure probe in a duct or wind tunnel. 

(j) The presence of a long connecting tube between pressure source and pressure pickup introduces 
errors in dynamic pressure measurement. 

Type D sources (the instruments themselves) produce errors that are usually negligible in comparison 
with those of the other types. The indication of a sensor used to measure a physical quantity can usually 
be obtained with very little error; but the relation between that indication and the physical quantity that 
the sensor is intended to measure is affected by sources of types B and C. 
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Examples of operations affected by type E sources are 

(a) Matching of brightness by eye when using a disappearing-filament type of optical pyrometer or 
a line-reversal type of pyrometer 

(b) Matching luminous intensity in visual photometry or nephelometry 

(c) Establishing coincidence of image positions when using an optical range finder 

(d) Perception of indicating-accelerometer pointer position during a controlled acrobatic airplane 
maneuver 

(e) Matching or detecting sound amplitudes in audiometry. 

3.3 Correction and error. The quantity that must be added algebraically to the observed indication y in 
in order to obtain the true value y tr is termed the correction Ay. The quantity (-Ay) is termed the error . 

(3.3-1) Correction = A y = y tr - y in 

(3.3-2) Error = e = - Ay = y in - y tr 

3.4 Range and span . For many instruments, the following terms are used partially to define instrument 
characteristics. 

Range: A statement of the lower limit y min and the upper limit of y that the instrument can 

measure 

Span: The absolute value | y^ - y min | of the algebraic difference between the minimum and 

maximum values of the range 

Full scale: The value of y^. The term is of practical utility only when the lower limit of the range 
is zero or when the limits of the range are ±ymax* 

3.5 Methods of describing the accuracy of a single measurement. The relation between the true and 
the indicated values of a quantity y may be described in several ways. Some of these are 

(a) by stating the value of Ay 

(b) by stating the fractional value Ay/y in (also called the relative value) or its equivalent percentage 
value 100 Ay/y in . When this choice of designation is made, the fractional error 

(3.5-1) E = e/y in 

is used in place of Eq. (3.3-2) to describe the error. 

(c) by using a combination of (a) and (b); for example, 

(3.5-2) Correction = Ay + m% of y 

(d) by stating the correction as a fraction of span 

4y/|)W -yminl 

(e) by stating the correction as a fraction of full scale 

Ay/ymax 


(f) by stating the ratio 


yjyi n = i + (Ay/yin) 
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The appropriate choice among these ways is usually deemed to be the one which is likely to have the 
same value over the entire range of the instrument. The nature of the instrument or of the measured quantity 
usually determines this choice. 
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3.6 Methods of describing the accuracy of an instrument. The accuracy of an instrument often can be 
described by providing a table or graph that gives the value of Ay, Ay/y in , Ay/span, Ay/y^*, or y tr /y in 
over the entire range of the instrument (unless one of these values happens to be constant over the entire 
range). These values describing accuracy may have been obtained by instrument calibration or testing, or 
may have been estimated by analysis of the design and construction of the instrument. 

However, one may prefer to characterize the instrument’s accuracy by using a single compact expression 
that states the limit of inaccuracy that has been experienced in past use of the instrument or that can be 
expected in future use. Such choice is common among instrument manufacturers in preparing a catalog 
description. 

The limit of inaccuracy is usually described in terms of the absolute value of Ay, Ay/y in , Ay/span, or 
Ay/ymax* A common synonym for this (positive) quantity is ‘‘limit of error/’ and hence it will be designated 
by the abbreviation “LE.” The selection of the form to be taken by the expression for limit of error is 
rarely arbitrary; just as indicated in Sec. 3.5, the form is determined by the nature of the physical variable 
being measured and by the circumstances under which the instrument is being used. Some simple forms 
are listed in the following section. 

3.7 Limit of error . Some commonly used forms for describing the limit of error are 

(a) LE = v, where v has the dimensions of y. This form is equivalent to [Ay | < v and is often used 
to describe the accuracy of a resistor, capacitor, burette, or weight. 

(b) LE = V percent of reading, where Kis a numeric. This form is equivalent to |Ay/y| < ( V( 100) and 
is often used to describe the accuracy of a power supply, spring, or clock. 

(c) LE = V percent of span, where f is a numeric. This form is equivalent to 
|Ay/ (ymax ~ Jmm)l < ( 1 00) and is often used to describe the accuracy of a thermocouple-type 
pyrometer, a multiple-range signal analyzer, or a sound-pressure meter. 

(d) LE = V percent of full scale, where V is a numeric. This form is equivalent to 
I Ay /ymax I < (V/100) and is often used to describe the accuracy of an ammeter or of a pressure 
gauge. 

(e) LE = v + (V% of reading), where v has the dimensions of y and Kis a numeric. It is often written as 

LE = v + V% 

This form is equivalent to |Ay| < (v + yF/100) and is often used to describe the accuracy of a 
resistance decade box (where v reflects the resistance of internal connecting wires and switch 
contacts), a dc microvoltmeter or potentiometer (where v reflects the presence of thermal emfs), 
a dynamometer (where v reflects the presence of friction), or an interval timer (where v represents 
starting and stopping errors). 

3.8 Nonlinear indications. The discussion of Secs. 3.4 through 3.7 has tacitly assumed that the instrument 
indication y is the same as the variable that is being measured, or is proportional to it. If the relation between 
the measured variable x and the indication y is nonlinear, the relation between Ax and Ay is obtained by 
appropriately differentiating the expression y = y(x). The forms of representation of inaccuracy that were 
presented in Secs. 3.5 to 3.7 may still be used, but will prove convenient only in certain cases: 

(a) If y oc x n , then, since 


dy/y = n 9 dx/x 

the value of any percentage correction to y is n times the corresponding percentage correction to x. 
For example, in the relation between kinetic pressure p, fluid velocity v, and density p 

p pv 2 , (Bernoulli’s principle) 

a 2-percent correction to a measured value of p implies a 1 -percent correction to the deduced 
value of v. 
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(b) If y oc log x as in Examples 2.8e, 2.8f, and 2.8/, then, since 

dy = dx/x 

a correction in the form of Ay becomes a correction in the form of Ax/x. 

For example, if LE of y = 0.03, then LE of x = 3%. 

Thus, in this case, the most convenient form of representing the correction to y differs from the most 
convenient form of representing the correction to x. 

3.9 Accuracy of an indirect measurement . If y is dependent on other variables x lt x 2 , x 3 ,... and is 
deduced from measurements of the x’s, then we may write 

(3.9-1) y = F(x,,j r 2 ,x 3 ,...) 

and the relation between the correction to y and the corrections to the x’s is 
dF dF dF 

(3.9-2) Ay = — A*! + — Ax 2 + — Ax 3 + ... 

OX | OX 2 uXy 

In order to determine the derivatives, it is not necessary that Eq. (3.9-1) be known as an analytic, 
differentiable formula. Since Ay is merely a small correction, dF/dx; need be known only approximately. 
Thus, 

(a) If a tabulation of y exists for various values of the x,, the value of dF/dx, may be approximated 
by Ay, /Ax, in the vicinity of the values of x it x 2 > x 3> ... at which 3F/dx, is desired. If the tabulation 
is sufficiently fine, Ay, and Ax, may be taken as the differences between adjacent tabular entries. 
If the tabulation is not sufficiently fine, the values of the increments may be deduced by quadratic 
interpolation among the tabular entries surrounding the point of interest. Sometimes, inspection 
of the entries will reveal that a retabulation of some function (e.g., the reciprocal, square, or 
logarithm) of the variables will facilitate interpolation. 

(b) An experiment may be performed in which x, is changed by an amount Ax,, while all other x’s 
are held unchanged, and the corresponding Ay, observed [Note N3.9], 


The various sources, A through E, listed in Sec. 3.2, usually represent separate variables. With this 
understanding, and if the A’s are, for the moment, assumed to represent inaccuracies in knowledge of the 
associated variable, then the relative magnitudes of the terms on the right side of Eq. (3.9-2) serve to indicate 
where any effort to improve accuracy (if such improvement is necessary) should be directed: 

(1) Terms of smaller magnitude represent operations or measurements that are already adequate. 

(2) Terms of larger magnitude represent operations or measurements where improved accuracy would 
be significant. 

The implications of Eq. (3.9-2) are presented more fully in Sec. 3.28 as Eqs. (3.28-3) and (3.28-4). 

3. 10 Systematic and random errors . Errors of measurement may generally be characterized as either 
systematic or random. 

Systematic errors are errors whose sign and magnitude are always the same under the same circumstances. 
If the sign and magnitude of a systematic error are known, the corresponding correction can be made to 
the measurement, so that the error is eliminated. If the nature of a systematic error is understood, but its 
magnitude is not known, deliberate steps may be taken to eliminate or to reduce its influence (Sec. 3.11). 
At the other extreme, the existence of a systematic error may not be suspected until a comparison with 
the results of another laboratory reveals a systematic difference in results, prompting a more intensive search 
for the systematic cause of the difference. 
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Random errors are errors of unknown sign, whose magnitude may only be estimated. If the random errors 
are of accidental origin, their magnitude may be estimated by making repeated observations under apparently 
identical circumstances. 

The distinction between systematic and random errors is not unequivocal. If one does not choose to correct 
for a systematic error, it may in effect become a random one (see examples c through f in Sec. 3.1). If 
only the sign and LE of a systematic error are known, one may apply (1/2) LE as a systematic correction 
and treat the remaining error as a random one whose limits are ±(1/2) of the original LE. Conversely, 
effective graphical and analytical techniques, like the method of least squares, are available that permit 
convenient partition of a measurement error into a systematic component and a random component. These 
techniques will be treated later; their selection and use involve a human choice among the convenience, 
complexity, and precision of describing the uncertainties. Furthermore, as one’s experience and understanding 
increase, the sign and magnitude of an originally random error may be learned, so that it becomes a systematic, 
correctable one. 

3.11 Systematic errors . Although an experiment may be attended by several systematic errors, only those 
errors need be considered whose magnitude may be sufficient to affect the result appreciably. Some expedients 
for dealing with systematic errors of uncertain but possibly significant magnitude are the following: 

A. Appropriate experimental techniques may serve to cancel the error or to reduce it to an acceptably 
small magnitude. 

B. If an experiment is intended to determine the effect upon y of one particular variable jcj, where 
jcj is only one of several independent variables, it may be possible to tolerate the existence of large 
systematic errors that do not depend on because these errors will remain constant throughout the 
experiment and can be canceled out of the final result. The experiment may be arranged deliberately to 
permit this situation by calling for the measurement only of dimensionless ratios. A considerable simplification 
of apparatus and technique and considerable improvements in speed, cost, and efficiency of operations are 
often realized by such a procedure. 

Examples of systematic errors will be recognized among the examples listed in Sec. 3.2. Other examples 
and techniques for the reduction of errors are listed here: 

Example 3.11a. The eccentricity of the divided drum of a micrometer screw produces a periodic, 
sinusoidal, error in the reading. As a correction, one may take the mean of the readings of two 
indexes 180° apart. 


Example 3.11ft. The eccentricity of an orifice plate in its holder produces an error in flow rate 
indication. However, (a) the same flow rate can be duplicated as often as desired without knowledge 
of the absolute value of flow rate, and (b) moderate changes in flow from some reference value 
(the set point) can be measured with negligible error. The ability to hold a set point and to correct 
for deviations from that set point, with only nominal knowledge of the exact flow rate, is a common 
requirement of many industrial processes. 


Example 3.11c. Inequality in the length of the lever arms of a chemical balance results in a 
proportionate error in weighing. As a correction, the method of substitution eliminates the error. 

Example 3. lid. Demagnetization of the permanent magnet in an electrobalance results in a 
proportionate error in weighing. As a correction, the method of substitution is applicable here. 
Furthermore, if the balance is accurately linear, the use of one accurate substitution weight will 
serve to establish the new proportionality constant. If the balance is slightly nonlinear, the use of 
two accurate substitution weights (one near full-scale value and the other near half-scale value) will 
also serve to establish the nonlinearity. 
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Example 3. lie. Self-heating of a resistance-thermometer bulb measuring the temperature of gas 
in a tank produces an error in temperature indication. However, repetition of the measurements, 
using 1/2 and 1/4 of the electrical power originally used, will permit 

(i) extrapolation to the indication that would be obtained if there were zero power dissipation 
in the bulb, and 

(ii) estimation of the optimum power dissipation that will provide the best compromise between 
self-heating error and sensitivity, to yield adequate accuracy with only a single measurement. 

Example 3.11/. Background radiation from the walls of the enclosure housing the components 
of an infrared spectrometer, and from those components themselves, may cause an error in spectral 
amplitude measurement. If a light chopper is used to chop or modulate the beam entering the 
spectrometer, at a fixed frequency, and the detector is sensitive only to signals at and near to the 
chopping frequency, the effect of the background radiation may be eliminated. 

Example 3.11g. Thermal expansion of an extensometer relative to the specimen produces a false 
indication of strain that is proportional to the temperature change. As a correction, the strain indication 
may be compared with that of a second extensometer attached to an unstrained specimen of the same 
material at the same temperature. Alternatively, the temperature may be monitored and a correction 
computed from the results of a separate experiment performed at zero strain and variable temperature. 

3.12 Random errors . Random errors may arise from such sources as 

(a) inability to read to better than the least count of a digitized display 

(b) parallax in reading pointer position 

(c) friction and backlash in a mechanical device 

(d) vibration of the instrument support 

(e) contact potentials, in measuring small electrical voltages 

(0 electronic noise 

(g) changes in temperature and pressure when their effects are not understood 

(h) uncertainty in the magnitude of systematic corrections 

(i) fluctuations in the physical variable being measured, as illustrated by many of the examples of 
type A sources listed in Sec. 3.2. 

The limit of error (LE) is a useful, convenient means of describing the capability of a single instrument. 
However, when there are several independent variables in Eq. (3.9-1), the individual LE’s may not be 
summed, because such summation would lead to an unrealistically pessimistic result whose occurrence has 
extremely low probability. A more realistic approach is to ask what is the most likely order of magnitude 
of the random error, or, more precisely, what is the probability that an error of given magnitude will not 
be exceeded. Answers to these questions use the language of statistics and require a study of repeated 
measurements of the physical variable of interest, or else the application of information obtained from 
previously performed repetitive measurements. The answers also depend on what intuitive assumptions are 
made concerning the nature of the random errors. 

It is this statistical approach which is the principal concern of the remainder of this chapter. For a particular 
set of assumptions and a particular group of repetitive measurements, it will be possible to establish 

(a) the most probable value of the measured quantity, 

(b) measures of how well the individual observations already made agree with that most probable 
value, and 

(c) measures of how well future observations are likely to agree with that most probable value, under 
the assumption that future conditions of measurement are the same as those that existed during 
the past measurements. 

The random error in y will often be written as 5y ; the presence of the ± sign in front of 5y is implicit 
in its definition. 
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3.13 Most probable value of a quantity . Means . The median. If repetitive measurements of a quantity 
y are made in order to reduce the effect of random errors, one may then ask what is the most probable 
value of y that is to be deduced from the measurements. A similar question may be asked if one seeks a 
single value which will conveniently represent an average of several values that differ systematically or 
randomly. The answer depends on the nature of the quantity v and on the method and purpose of its 
measurement. 

If V|, y 2 ,... ,y„ are the n measurements of y and v 0 is the average or “mean” that is sought, some 

commonly used definitions of y 0 are the following: 

A. The arithmetic mean y 0 is defined by 


n 

(3. 13- la) n»y 0 = £ y, 

l 

It is used when the individual measurements are considered to be of equal importance (i.e., when, a priori , 
the likely absolute error <5y, of a measurement is considered to be the same for all y ; ). 

The corollary weighted arithmetic mean y 0 is defined by 


(3. 13-lb) y 0 »^ w t = wpj 

l 1 

It is used when the individual measurements are considered to be of unequal importance and their relative 
importance is represented by their respective weights w r Some suggestion of how weights may be assigned 
will be given in Sec. 3.21. 

B. The geometric mean y 0 is defined by 


(3. 13-2a) y% = y x y 2 . ,.y r ,.y n = y, 

1 

All the y, must be positive. This relation is tantamount to taking the arithmetic mean of the logarithm of 
the y*s. 


n 

(3 . 13— 2b) /i* log y 0 = £ log X 

l 

The quantity y Q in this expression has sometimes been called the logarithmic mean . The logarithm may 
have any base, including the Napierian and the common. 

The geometric mean or the logarithmic mean is most appropriate when, a priori , the likely fractional 
error 6>v/y, of a measurement is considered to be the same for all y,. 

C. The /nth root y 0 of the arithmetic average of the /nth power of y is defined by 


(3. 13-3a) n»y = £ >f 
1 


A special case is the harmonic mean y 0 , defined by 
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n 

(3. 13— 3b) n/y 0 = £ (l/j,) 

1 

Another special case is the root-mean-square (rms) value y 0 defined by 


n 

(3.13-30 n*Vo = £ vf 
i 

D . In some cases, it may be appropriate to compute y 0 from the arithmetic mean of some other 
function of y. Thus, the arithmetic mean of the logarithm was taken in B . 

E. The median of n measurements of y is that value y 0 such that half of the measurements are less 
than jo and half are more than y 0 (if n is even, y 0 lies halfway between two measured values). It is useful 
when the individual measurements are not considered to be of equal importance, but their relative importance 
is not known, a priori. 

Just as one may take the arithmetic mean of powers, logarithms, or other functions of y in order to determine 
the most probable value y 0 , so may one, alternatively, take the median of powers, logarithms, or other 
functions of y. 

All of the above formulas would yield the identical value of y 0 if all of the y, were identical. Consequently, 
when all of the y t are almost alike (i.e., when the 6y, or Sy,/y ( are very small) one may choose, with 
insignificant loss of precision, a mean (or median) which is conveniently calculable or convenient for 
mathematical analysis, rather than the mean which is academically more appropriate. 

Example 3.13a. Gas flows through a duct; gas pressure and linear velocity are constant across 
the cross section, but gas temperature varies with location in the cross section. In order to deduce 
mass flow rate from the equation 

mass flow rate = area x linear velocity x mean density, 
the harmonic mean of the absolute temperature is required. 

Example 3.13b. Gas flows through a duct; gas pressure p and temperature T are constant across 
the cross section, but gas velocity, as deduced from the pressure differential Ap developed by a 
pitot-static tube, varies with location in the cross section. In order to deduce mass flow rate m from 
the equation 

m = \[2p*Apf (RT)] XI2 dA 

where R is the specific gas constant and dA is the element of area, an area average of V(Ap) is 
required. 

Example 3. 13c. To determine the average decibel level over an area, sound pressures are measured 
at several appropriate locations. The decibel level is then computed from the geometric mean of 
the pressures. 

Example 3.13d. The following mean values exist in a high-vacuum system in which there is free- 
molecule flow (a high-vacuum enclosure is defined as one whose dimensions are much smaller than 
the mean free path of the gas molecules): 

(i) The mean free path of the molecules is the arithmetic mean of the lengths of the paths 
of gas molecules between successive collisions with other molecules. 

(ii) The arithmetic mean speed of the molecules. It determines the volumetric rate of flow of 
gas through a constriction like an orifice. 

(iii) The rms speed of the molecules. It determines the pressure of the gas at any location; in 
particular, it determines the pressure on the walls bounding the system. 

[Note N3. 13] 
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3.14 Residuals . In any experimental measurement that is not perfectly accurate, the true value y tr of the 
quantity of interest is not known; instead, the most probable value y 0 is deduced as the mean (or median) 
of a number of repetitive measurements of >\ perhaps by processes like those illustrated in Sec. 3.13. 

If a series of measurements y, lead, by whatever process, to a most probable value y 0 , one defines the 
residual v, as 

(3.14-1) v, = y, - y 0 

The most probable value y 0 and the residual v can be calculated from the measurements y,; the error 
e cannot be obtained from Eq. (3.3-2) because y tr is not known. Worthing (Ref. 3-1) has pointed out that, 
in most engineering measurements, the distinction between v and e is academic, because the difference 
between useful quantities derived from them is smaller than the uncertainty in knowledge of either quantity. 
However, the use of the residual is so firmly entrenched in the literature, that the distinction will be maintained 
in this text. 

It is also sometimes convenient to deal with the fractional residual 
(3.14-2) V = (y, - y 0 )/y, 

3.15 Statistical descriptions of the past and the future. Past events can be described with all desired 
accuracy, simply by listing all the data. However, when there are many data, it may be possible to provide 
an adequate description by the shorthand of statistical terminology; such condensed description invariably 
implies that certain assumptions have been made about the nature of the data. The descriptions and the 
assumptions are approximations to the truth, and are justified by the convenience of a concise representation 
of the results and the convenience of any subsequent analysis of the results. 

Future events cannot be predicted; they can only be estimated. The estimation is made by examining 
the statistical description of past events and assuming that the conditions that produced the past events will 
remain unchanged in the future, so that future performance can be an extrapolation of past performance. 

Two questions of interest to those who use statistical terminology to describe a measurement are 

1 . How well are past data represented by their shorthand statistical description? 

2. How well is the statistical description estimated to represent future results? 

Answers to these questions depend on the assumptions made concerning the nature of the errors. These 
assumptions will be a priori assumptions because they must precede the data analysis and interpretation. 
In this text, only a few of the more common assumptions will be treated. 

One of these is the nature of the random-error distribution. (Three common distributions will be described 
in Sec. 3.18.) Any assumed type of distribution will then be characterized quantitatively by the dispersion 
of the data that have been acquired. The name of the distribution merely identifies its shape; the dispersion 
describes numerical magnitudes that characterize that shape. The assumption of a distribution shape precedes 
the data analysis; the dispersion is a result of that analysis. 

Another assumption, made a priori , is whether all data are likely to have the same uncertainty (error, 
by), or to have the same fractional uncertainty (fractional error, by/y), or whether each error is likely to 
have a weight w. The choice among these likelihoods is merely a choice of the form of the error, not of 
its numerical magnitude. It is made on the basis of prior experience and understanding of the nature of 
the physical quantity being measured and of the apparatus, techniques, and circumstances of the measurement 
operation. For example, an assumption that by/y is constant will place greatest weight on points near y = 0 
and cannot be made if y may pass through zero. If the error is thought likely to be of the form by + Ey 
(Eq. (3.5-2)), with both by and Ey of comparable magnitude, the use of w is required. 

If a logarithmic scale of ordinates would be most appropriate to a graphical presentation of the data, then 
an a priori assumption that by/y = a constant is usually indicated; if a uniformly divided scale of ordinates 
would be most appropriate, then an a priori assumption that by — a constant is usually indicated. 

3.16 The random-error distribution. Given a single instrument indication expected to represent the value 
of some physical variable, any one indication, after application of all known systematic corrections, may 
not represent the true value of the variable because of the presence of random errors. Because of such random 
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errors, it will never be possible to know the true value of the quantity. However, by repeated performance 
of the measurement operation, the totality of readings may be analyzed mathematically to establish the most 
probable value of the quantity; furthermore, this most probable value may be made to approach the true 
value by a sufficient increase in the number of repetitive observations. The extent to which the repetitious 
measurements need to be made depends on how accurately the quantity must be known; the extent to which 
the analysis can be carried depends on how well the law governing the distribution of errors is known. 

A common assumption concerning random errors is that positive and negative errors are equally likely. 
This assumption will be made in the following text. 1 

Consider that a very large number n of repeated measurements y x ,y 2t ... ,y„ of the same quantity 

y XT possesses random errors whose absolute magnitudes are e h e 2 ,.. .,e n respectively. (Symbol e is 
thereby a shorthand symbol for \e\.) Let the largest possible value of these errors be e m . The smallest 
possible value is zero. Divide the range of values of error into a number of equal intervals Ae f chosen 
to be small and yet to include a reasonably large number of errors in most of the intervals. Plot the number 
of errors that lie in each interval versus the value of the error at the midpoint of that interval. Such a plot 
is termed an error-frequency distribution . 

As n^oo to the extent that the concepts of the differential calculus may be used, a curve f(e) may be 
faired through the points. As Ae— 0 because n— oo, the probability dP that an error lies in the interval de is 

(3.16-1) dP = f(e) % de 

The probability that an error lies between the limits e a and e b is 
(3. 16-2a) P = l b f{e)*de 

In particular, the probability that an error lies between 0 and e m is a certainty, so that 
(3. 16— 2b) l m f(e)*de = 1 

Jo 

The function 

(3.16-3) f(e) = dPfde 

is termed a probability coefficient or a frequency distribution function or just an error distribution. Note 
that /(*)•<& is dimensionless. Note also that, by Eq. (3.16-1), and the principles of the differential calculus, 
the probability of occurrence of any one error e is zero; only the occurrence of a band of errors, like 
e a < e ^ e b , as in Eq. (3.16-2a), has a nonzero probability. 

3.17 Concise idealized measures of random-error dispersion . For the most common error distributions, 
three measures of dispersion have been widely accepted. These are the probable error, the average deviation, 
and the rms error (standard deviation). Since the assumption has been made that positive and negative errors 
are equally likely, and with symbol e representing \e\, we define, for 0 < e < e m and n — oo, , 


■lf an error distribution is not symmetrical about the abscissa e = 0, it is said to be skewed. This asymmetry will be touched on 
briefly in the discussion of correlation coefficients (Sec. 4.12) but otherwise will not be treated. 
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(3.17-1) 

(3.17-2) 

(3.17-3) 


A 0 = 
A i — 

A 2 = 



f(e)*de 
e*f( e ) *de 
e 2 *f(e) m de 


We then have the following mathematical definitions of the three measures of dispersion of any form 
of error distribution. 

The probable error (e p or p or p.e.) is the abscissa which divides the curve f(e) vs e into two equal areas. 
(3.17-4) ^Ae)»de = A 0 /2 

The average deviation (e a or a or a. d.) is the abscissa which is the center of moment of area of the curve 
/(e) vs e. 

(3.17-5) e a = A\/Aq 

The rms error (e a or o or standard deviation) is the abscissa which is the center of the second moment 
of area of the curve f(e) vs e. It is the positive square root of the variance which is given by 

(3.17-6) el = A 2 /A 0 

When the number of observations is so large that f(e) may be treated as a continuous function for all 
practical purposes, alternative phrasings of these definitions are 
The probable error e p is such that half of all the errors (in absolute magnitude) are smaller than e p and 
half of the errors are larger than e p . 

The average deviation e a is the arithmetic mean of the absolute values of all the errors. 

The rms error e a is the square root of the arithmetic mean of the squares of all the errors. 2 

3.18 Gaussian , uniform , and median error distributions . Some common random-error distributions will 
be identified here. It is assumed that all measurements have been corrected for known systematic errors. 

/. The idealized Gaussian distribution is the result of the following assumptions: 

A. Positive and negative errors are equally likely. 

B. All magnitudes of error are possible. 

C. The frequency of occurrence of an error depends on its magnitude. 

D. Large errors are less likely than small ones. 

E. The likelihood of very large errors is very small. 

F. All measurements have been made with equal care. 

G. The number of measurements is large enough to warrant the application of the differential calculus. 


2 One of the uses of these measures of dispersion is to establish whether two values of a quantity are significantly different, when 
each value represents the mean of a number of observations. Each mean has its own measure of dispersion. If the ratio (> 1) between 
the two measures is less than 1.5 to 2, it is unlikely that the difference between the two values is significant. This matter is treated 
a little more precisely in Sec. 3.22. 
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Then if, a priori, each value of y is deemed likely to have the same random error 6v t 

(a) the most probable value of y is the arithmetic mean y 0 (Eq. 3.13-la); 

(b) the error distribution f(e) [Eq. 3.16-3] is shown in Fig. 3.18(a) and is given by the equation 
(3.18-1) f(e) = (1/<j)(1/V2^) exp [-e 2 l(2a 2 ))\ . 

(c) the idealized measures of dispersion (Sec. 3.17) are in the ratios 
(3.18-2) e p : e a : e a = 0.6745: V(2/tt): 1 

and 

(3.18-3) e 0 — o. 



TABLE 3.18.1. -RELATIONS AMONG ERRORS 
OF THE GAUSSIAN DISTRIBUTION 
Value of Y/X 


Y 

X 


e p 


*<7 

e 90 

e 95 

e P 

1 

0.845 

0.674 

0.410 

0.344 


1.183 

1 

V(2/r) 

0.485 

0.407 


1.483 

V(t/2) 

1 

0.608 

0.510 

e 90 

2.44 

2.06 

1.64 

1 

0.84 

e 95 

2.91 

2.46 

1.96 

1.19 

1 




(a) Gaussian. 

(b) Uniform. 

(c) Median. 

Figure 3.18 — Error distributions. 
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TABLE 3. 18.2 -ERRORS CORRESPONDING TO VARIOUS PROBABILITIES AND ODDS 


[Errors are expressed as multiples of probable error p, average deviation a, or rms error a.) 


c m 

P+ 

Gaussian 

distribution 

r-distribution 
e/o - f/[(£v 2 )/w] ,/2 

Uniform 

distribution 

Median 

distribution 

e/p 

eia 

e/o 

e/p 

e/a 

e/o 

e/p 

e/a 

e/o 

n = 16 

n = 8 

n = 4 

1:1 

50. 

1 


0.67 

0.69 

0.71 

0.77 

1 

0.87 

1 

0.69 

0.49 

1:1.4 

42. 


1 










1:2.1 

32. 



1 









1:4 

20. 



1.28 

1.34 

1.42 

1.64 

1.6 

1.39 

2.32 

1.61 

1.14 

1:4.6 

17.7 

2 











1:5.7 

15. 



1.44 

1.52 

1.62 

1.92 

1.7 

1.47 

2.74 

1.90 

1.34 

1:8 

11. 1 


2 










1:9 

10. 



1.64 

1.75 

1.90 

2.35 

1.8 

1.56 

3.32 

2.30 

1.63 

1:19 

5. 



1.96 

2.13 

2.37 

3.18 

1.9 

1.65 

4.32 

3.00 

2.12 

1:21 

4.55 



2 









1:22 

4.3 

3 











1:32 

3. 



2.17 

2.40 

2.71 

3.90 

1.94 

1.68 

5.06 

3.51 

2.48 

1:49 

2. 



2.33 

2.60 

3.00 ' 

4.54 

1.96 

1.70 

5.64 

3.91 

2.77 

1:59 

1.67 


3 










1:370 

0.27 



3 










•Odds that a greater error will occur 
+ Probability of exceeding the tabulated error, percent 


(See Footnote 3.) 

The relations among the various errors are indicated in Table 3. 18.1, which also includes errors e^Q and 
e 95 that are exceeded only 10 or 5 percent of the time, respectively. Table. 3. 18.2 lists the probability P 
that an error will exceed a given multiple of the common measures of dispersion. The percentage (100-P) 
has been termed a confidence limit , and has been used to indicate the reliability of an estimate of future 
results. The information on probabilities is restated in an alternate form, in Table 3.18.2, by giving the 
odds C of exceeding a given magnitude of error. 

In practical engineering, the Gaussian distribution may be truncated because observations whose deviations 
from the mean exceeded some value like 3e a or 3e a were rejected as grossly unreliable. (A common cause 
is a typographical error.) Such truncation has negligible effect on most of the ratios shown in Table 3.18.1 
and in Eq. (3.18-2), although the magnitudes of the measures of dispersion (as in Eq. 3.18-3) are reduced 
by the rejection. 

[Note N3. 18] 

//. The idealized uniform distribution is the result of the following assumptions: 

A. Positive and negative errors are equally likely. 

B. The absolute magnitude of an error cannot exceed e m . 

C. For \e\< e m , the frequency of occurrence of an error is independent of its magnitude. 

D. All errors are equally likely, as long as \e\ < e m . 

E. All measurements have been made with equal care. 

F. The number of measurements is large enough to warrant the application of the differential calculus. 


3 A fourth measure of dispersion, h = l/(o V2), called the index of dispersion, is used less frequently in present-day engineering 
applications of the theory of errors. 
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Then if, a priori, each value of y is deemed likely to have the same random error by, 

(a) the most probable value of y is the arithmetic mean y 0 (Eq. 3.13-la); 

(b) the error distribution /(e) [Eq. 3.16-3] is shown in Fig. 3.18(b) and is given by the equation 

(3.18-4) /(e) = l/(2e m ) M ^ e m ; 

(c) the idealized measures of dispersion (Sec. 3.17) are in the ratios 
(3.18-5) e p : e a : e„= I: 1: (2/V3) 

and 

(3.18-6) e a m a - e m /2 

The probability P and the odds C of exceeding a given error are listed in Table 3.18.2. 

III. The idealized median distribution is the result of the same assumptions as those for the Gaussian 
distribution, except that previously stated assumption F is replaced by 

F. All measurements were not made with equal care and their relative reliability is not known in 
advance. 

Then 

(a) the most probable value of y is the median (Sec. 3.13..E.) 

(b) the error distribution f(e) [Eq. 3.16-3] is shown in Fig. 3.18(c) and is given by the equation 
(3.18-7) f(e) = (1 la) exp(-|e|/n) 

(c) the idealized measures of dispersion (Sec. 3.17) are in the ratios 
(3.18-8) e p : e a : e„ = In 2: 1:V2 

and 

(3.18-9) e fl =a 

IV. In any of these distributions, if, a priori, each measurement of y is not deemed likely to have the 
same random error but is deemed likely to have a different random error 6y„ then y should be replaced 
by an appropriate function of y that may be presumed to have constant error. In particular, 

(1) If y is deemed to have constant fractional error 5y/y then y should be replaced by its logarithm; 
y 0 becomes the logarithmic mean or median of the data (Sec. 3. 13), and e is replaced by E, the 
nondimensional fractional error (Eq. 3.5-1). 

(2) If v is deemed to have a weight w, then y should be replaced by wy, the quantity yo becomes 
the weighted mean (Eq. 3.13-lb) or weighted median, and e is replaced by e/w. (Weighted 
observations will be treated in Secs. 3.19 and 3.21). 

3.19 Practical measures of random-error dispersion. The names and definitions of Sec. 3.17 are so 
convenient that they are used to describe the dispersion of data even when the number n of measurements 
is small and /(e) does not have the analytic character implied in Eqs. (3.17-1 to -3). (Analytic character 
means that /(e) is continuous and has a continuous derivative.) The integrals are replaced by summations, 
the errors are replaced by residuals, and the formulas are modified to allow for the fact that the most probable 
value y 0 derived from repetitive measurements y, (t = 1,2 n) is not the true value y u , which is never 
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known. The actual formula for y 0 depends on the assumptions made concerning the nature of the errors, 
but the following terms and symbols usually apply. 

The measures of error fall into two groups. Terms in one group, whose descriptions contain the words 
“of a single observation,' ’ pertain to past events. The terms describe how well the individual data already 
taken agree with the most probable value y 0 of the measurements. The numerical value of these measures 
of dispersion is almost independent of the number n of measurements that have already been made. 

Terms in the second group, whose descriptions contain the words “of the mean" or “of the median/’ 
are useful for future use. The terms describe how well y 0 is likely to represent the true value y Ir of the 
measured quantity. This likelihood will increase as the number n of past contributing measurements increases; 
the numerical value of these measures of dispersion will usually be found to vary inversely with Vn. The 
terms in this second group provide estimates of the likelihood that y 0 will represent y tr in the future, 
assuming that there will be no change in the circumstances governing the measurement. 


Case A. Equally-weighted measurements . Given n observations y i (i = 1,2 n ) such that, a priori , 

all V/ are deemed to have equal uncertainty 5v, the following definitions hold as substitutes for the equations 
of Sec. 3.17. (All summations are from i = 1 to / = n.) 

The most probable value of y is the arithmetic mean 

(3. 13- la) y 0 = (Ey,)/n. 

If the residuals are defined as 


(3.14-1) 

v, = y t - y 0 

then 


(3.19-1) 

e a \ = (E|v, |)/[/i(n - 1)] 

(3.19-2) 

= ^(Jo) = e a\/^ n 

(3.19-3) 

e 2 a] = (Ev?)/(n - 1) 

(3.19-4) 

?o0 = e„(yo) = e^/'Jn 

(3.19-5) 

*po = e p (y 0 ) = <? pl /Vn 

where 



e aX average deviation of a single observation 

e^Q average deviation of the mean 

e a \ rms deviation of a single observation 

e o0 rms deviation of the mean 

e p \ probable error of a single observation 

epo probable error of the mean 

The probable error of a single observation is, in theory, that value, e pX , such that half of the residuals 
(in absolute magnitude) are less than e pX and half are greater. In practice, the value of e p] is deduced from 
e a \ or e aX , by a formula that depends on the assumed distribution of errors. For example, for a Gaussian 
distribution, by Eq. (3.18-2), 

(3.19-6) e p] = 0.61e aX = O.S5e aX 
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and, for the uniform distribution, by Eq. (3.18-5), 

(3.19-7) e p] = e al = eJ2 - 0.87e ffl 

The dimensions of the e x 's, the quantities on the left side of Eqs. (3.19-1 to -5), are the same as the 
dimensions of y,. 


Differentiation of Eq. (3.19-3) shows that making Evfa minimum leads to the definition of y 0 given by 
Eq. (3. 13- la). Conversely, the most probable value y 0 may be deduced by finding the condition that will 
minimize Evj. This procedure of finding y 0 is thereby termed the method of least squares. 

Case B. Measurements with constant fractional error. Given n observations y, (i = 1,2 n) such 

that, a priori, all y, are deemed to have equal fractional uncertainty 5y/y, the following definitions hold 
as substitutes for the equations of Sec. 3.17. (All summations and continuous products are from i = 1 to i = n. ) 

The most probable value of y is the geometric mean y 0 , which is also the antilog of the arithmetic mean 
of the logarithms of the y,. If one defines the nondimensional residual 

(3.19-8) Vj = Vj/y, =(y,-y 0 )/y, 

then the following nondimensional quantities (fractional errors) are useful. 

(3.19-9) E a] = (E|^|)/[/i(n - l)] l/2 

(3.19-10) E a0 = E a (y 0 )=E al /Jn 

(3.19-11) El, = (LVf)/(n - 1) 

(3.19-12) EgQ = £„(y 0 ) = £„i/Vn 

(3.19-13) E p o = E p (y 0 ) = E^/Vn 

where 

E a i average fractional deviation of a single observation 
E a o average fractional deviation of the mean 
E o] rms fractional deviation of a single observation 
E a o rms fractional deviation of the mean 

probable fractional deviation of a single observation 
Epo probable fractional deviation of the mean 

The probable fractional error of a single observation is, in theory, that value, E p ,, such that half of the 
values of Ml are less than E p[ and half are greater. In practice, the value of E pl is deduced from computed 
values of E a ] or E ai by formulas that depend on the assumed distribution of errors. For example, for the 
Gaussian distribution, by Eq. (3.18-2), 

(3.19-14) E pi = 0.61 E„i * 0.85£ aI 

and for the uniform distribution, by Eq. (3.18-5), 

(3.19-15) E p i = E al = EJ2 * 0. 87£ al 
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Case C. Unequally-weighted measurements. Given n observations y, (i = 1 ,2,... ,n) such that, a 
priori , each y^ is deemed to have a weight w it the following definitions hold as substitutes for the equations 
of Sec. 3.17. (All summations are from / = 1 to i = n. ) 

The most probable value of y is the weighted arithmetic mean y 0 

(3. 13- lb) y 0 = [HwMmwf 

If one defines the residuals 

(3.14-1) v, = y, - y 0 

then the following definitions hold. 

(3.19-16) e a] = [£(|v,|V»v,. )]/[«(« - 1)] 1/2 

(3.19-17) e M = e a (y 0 ) = e^/'J'Lw, 

(3.19-18) e 2 ai = [E(vfo ,)]/(« - 1) 

(3.19-19) e M = e„{y 0 ) = eJ'JLw, 

(3.19-20) = e p (y 0 ) = ^,/VEw, 

where 


e Q \ average deviation of a single observation of unit weight 

e ao average deviation of the weighted mean 

e al rms deviation of a single observation of unit weight 

e a0 rms deviation of the weighted mean 

e p \ probable error of a single observation of unit weight 

€pQ probable error of the weighted mean 

The probable error of a single observation of unit weight is, in theory, that value, e pX , such that half 
of the products |v, | Vw, are less than e pX and half are greater. In practice, e p{ is deduced from computed 
values of e a] or e ai by formulas that depend on the assumed distribution of errors. For example, for the 
Gaussian distribution, by Eq. (3.18-2) 

(3.19-21) e pl =0.67 e al = 0.85 

and for the uniform distribution, by Eq. (3.18-5), 

(3.19-22) e pl = e Q \ = e m !2 - 0.87 e al 

The important measures of unequally weighted measurements are e and e ^>. The other measures, 
e a \* e a \ » an d e p \ are merely convenient intermediate steps in the computation of e e^, and e^. 

It is noteworthy that assignment of a weight w t = 1 /yj is tantamount to solving the case of measurements 
with constant fractional error, so that the simpler treatment of Case B is usable. However, when the estimated 
error is of the form of Eq. (3.5-2), the treatment of Case C must be used. 

For easier comparison, Table 3.19 lists the principal formulas of this section. 

[Note N3.19] 
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TABLE 3. 19. -SUMMARY OF PRACTICAL MEASURES OF DISPERSION 




A priori assumptions 

Line 

Item 

6y = constant 

by/y = constant 

y, has weight w i 

1 

most probable value 

>'0 = (£>’,)/« 

In y 0 = (E In y,)/n 

y Q - 

2 

residual 

v , = >v - y 0 

£ 

1 

ll 

v , = v, - >0 

3 

errors of 

e pl see line 9 or 10 

E p , see line 9 or 10 

e p \ see line 9 or 10 

4 

individual 

e a[ - <E|v,-|)/[n<n - 1)] |/2 

E a , = (£|KM"<» - i)] l/2 

e a \ = [E(|v,fVw ( )]/[n(n - ])] l/2 

5 

observations 

e*\ = (Ev2)/(n - 1) 

El 1 = (LVf)l(n - 1) 

- 1 ) 

6 

errors of 

e p0 = e pl^ n 

E p0 = E p |/Vn 

£ 

11 

'V. 

< 

7 

the 

e M = 

E m = E al /y/n 

Li 

> 

ll 

■5 

8 

mean 

e „o = tjyln 

E c 0 = EJ'Jn 

'.0 = 

9 

Gaussian distribution 

e p , = 0.67^, - 0.85f a , 

E p , = 0.67E,, = 0.85E„| 

= 0.67e al « 0.85e pl 

10 

Uniform distribution 

e pi = 0.87e„, = e o] = tJ2 

E pl = 0.87E,, = E al = EJ2 

e p\ = 0 87f .l = e a\ = f m /2 


Some examples of the practical application of error distributions are 

Example 3.19a. The probable error of the values in a five-place table of logarithms is 0.000 0025. 

Example 3.19 b. Mass production of electrical resistors yields a Gaussian distribution of resistance 
values, centered around the nominal or intended value. “Precision resistors” are obtained by selecting 
from the lot those resistors whose measured value falls within a limit of error (LE) of 0 . 1 percent. 
The error distribution among this group is substantially uniform, because the distribution represents 
the peak of the Gaussian curve. Another group, which has errors less than 1 percent, may be selected 
from the remainder, for sale at a lower price. The error distributions of this group will be Gaussian 
except for a trough between — 0.1 and +0.1 percent. Presumably, for economic reasons, the e a 
of this group will be less than 0.5 percent so that there will be few discards. It is noteworthy that, 
where such manufacturing procedure is followed, all resistors have comparable stability; only the 
value of resistance may be in error (by less than 0. 1 percent for the “precision” group; by not less 
than 0.1 percent for the remainder). 

3.20 Percentiles An informative description of the dispersion of acquired data is the statement of the 
error magnitude which was not exceeded by most of the data. No particular error distribution need be implied. 
For example, one may give the 90th percentile (ninth decile) e 90 or the 95th percentile (nineteenth vigintile) 

or one may simply state that “A/ percent of the deviations from the mean value y 0 were less than e^. ” 
Fractional errors may be treated similarly. 

3.21 Weighting of observations . If a number of observations y, is made of a quantity y and each 
observation has a random error, then, after all systematic corrections have been made, the most probable 
value of y is the weighted arithmetic mean of the observations, where the weight w, to be assigned to any 
observation y, is inversely proportional to the square of the random error of that observation. Equation 
(3.13-lb) may then be applied. 

The assignment of weights requires only an assumption of the relative magnitudes of the random errors 
of the individual y r If there is no previous information available, then a priori assumptions are required, 
as indicated in Sec. 3.16. If the y, represent values determined by previous repetitive measurements, so 
that each y, already has an associated probable, average, or rms error, then the individual weight w, is to 
be taken as the reciprocal of the square of that error. 
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Example 3.21 a. If all measurements have the same uncertainty by, then w, may be taken as unity. 

Example 3.21 b. If all measurements have equal fractional error by/y, then by = y x a constant 
and w, may be taken as My] in order to determine the arithmetic mean of y. However, all 
measurements have equal weight in determining the geometric mean or the logarithmic mean of y. 

Example 3.21 c. If y = au 2 and u has constant uncertainty bu , then 

by = 2 au m bu = 2*<5w*V (ay) = Vy X a constant 
so that w, may be taken as 1 /y, in order to determine the arithmetic mean of y. 

Example 3.21 d. If y = au 2 and u has constant fractional error bu/u, then 

by/y = 2 •bu/u = a constant 


so that 

(i) w, may be taken as 1 fy 2 in determining the arithmetic mean of y 

(ii) w, may be taken as unity in determining the geometric mean or the logarithmic mean of y. 

Example 3.21e. The thermal conductivity /cq, at STP, of H 2 gas taken from the same storage 
cylinder is determined by three independent laboratories. They respectively report the values 
0.0167 ± 0.0003, 0.0169 ± 0.0002, 0.0168 ± 0.0004 W/(m # K), where the quantitites after the ± 
sign represent probable errors. Then the most probable value of k$ is 

0.0001 [(167/9) + (169/4) + (168/16)]/[(l/9) + (1/4) + (1/16)] = 0.01683 
with a probable error of 0.00006. 

(A problem of this type is treated in greater depth in Sec. 3.32.) 

3.22 Uncertainty in the practical measures of dispersion. The measures of dispersion listed in Sec. 3. 19 
are obtained by using a summation of residuals as an approximation to the integration of errors. These 
approximations themselves have uncertainties. Consequently, if e x (or E x ) represents any one of the errors 
tabulated in Sec. 3.19, the more precise statement of e x or E x is 

(3.22-1) (e x or £*) • [ 1 ±0/(n - IT] 

where 0 and y are numerics. Their magnitudes depend on the nature of the error distribution. The magnitude 
of /? also depends on whether 0/ (n — 1 ) 7 is intended to represent the probable, average, or rms fractional 
error of e x or E x ; these three magnitudes will be written as ft p , & af and /3 a , respectively. 

Expression (3.22-1) shows that the validity (not the values) of the various e’s is diminished as the number 
n of observations is diminished. It also serves to establish whether the distinction between probable, average, 
or rms errors ( e p , e a , or e 0 ) is significant or merely academic, and to define the number of significant figures 
that are warranted in the statement of e x or E x . 

For both the Gaussian and the median distributions, the fractional uncertainty in e x or E x is 

(3.22-2) & x N{n- 1) 

where 

(3.22-3) fi p : (3 a : 0 a = 0.67: V(2/t): 1 
However, for the Gaussian distribution 
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(3.22-4) 0„ = 1/V2 

and for the median distribution 

(3.22-5) j5 0 - l . 

The ratios in Eq. (3.22-3) are the same as those in Eq. (3.18-2) because the distribution of the “errors 
of the errors” is itself Gaussian for both the Gaussian and median error distributions. However, the uncertainty 
with which any measure of dispersion is known is larger for the median distribution than it is for the Gaussian 
distribution. 

Example 3.22. Nine measurements are made of a quantity believed to follow a Gaussian error 
distribution. 

(i) if e a * 0.1, the representation of e a by two decimal digits is justified. 

(ii) if e a « 0.8, only a one-decimal-digit representation of e a is justified. 

3.23 Rejection of observations. When a particular observation deviates from the mean by an amount 
that is several times the probable, average, or rms error, the decision on whether to reject it as spurious 
is a matter of human judgment. It does not follow from any mathematical laws. However, once the human 
judgment is made, it may be expressed in formal mathematical language; such formal expression is often 
useful if it is to be incorporated into a computer program or if the data are to be used by others less expert 
in understanding of the physical situation. 

The opinions of others are useful in formulating a criterion for rejection of observations. Many such opinions 
have been published. They usually assume a Gaussian error distribution and usually express the criterion 
as a multiple of one of the standard measures of dispersion; often the multiple is itself a function of the 
number n of observations. Most choices lie between 3e a and 3e a of the data. Since the criterion is merely 
a formal mathematical description of one’s intuition, an understanding of the circumstances affecting the 
measurement— apparatus, technique, and observer— is just as important as are the numerical data. 

After an observation has been rejected, the measures of dispersion must be recomputed. If the criterion 
for rejection is based on a measure of dispersion, it may be necessary to repeat the computation to preserve 
the formality. However, all such operations should be performed with full appreciation of the uncertainties 
indicated in Sec. 3.22. 

3.24 Complete statement of the value of a physical quantity . A complete statement of the value of v 
requires a statement of the probable value y 0 and of the estimated inaccuracy using any of the forms 
indicated in Sec. 3.5. Some common forms of the complete statement are 

(a) y 0 ± e x where e x is the error of the mean, in units of y 0 ; 

(b) y 0 O ± E x ) where E x is a numeric representing the fractional error of the mean; 

(c) y 0 ± (e x + lOQEj %). Although not mathematically correct, this expression is accepted as a 
convenient representation of the more rigorous but more awkward expressions 

(y 0 ± e x )(l ± E x ), or 

y 0 (1 ± E x ) ± e x . 

Traditionally, e x and E x represented probable errors in the English-language literature and represented 
rms errors on the European mainland. However, the use of rms errors has now become more common 
everywhere. 

Some writers have preferred to place the limit of error (LE) after the ± sign, usually defining the LE 
as the 95th percentile, although limits as high as 99.7 percent have been used. Because of these ambiguities, 
it is necessary that each writer explicitly state what type of error is represented by the entry after the ± sign. 
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3.25 Truncation and rounding . The uniform distribution is significant when one deals with the expression 
of a quantity to m decimal places when (a) it is known that a more accurate expression would require more 
than m decimal places, and (b) the (m + l)th decimal is not known. Two types of situation which lead either 
to rounding or to truncation are illustrated by the following examples. 

Example 3.25a. A four-place table of a transcendental function, like the logarithm, represents 
a rounded value. The tabulated value is the most probable value; its limit of error is 0.00005 and 
its probable error is 0.000025. 

Example 3.256. The indication of a commercial digital meter usually represents a truncated value. 
Assume that the indication is given to three decimal digits. Then the limit of error is almost —0.001 
and there is a 50-percent probability that the error is less than —0.0005. However, if the most probable 
value is taken to be 0.0005 more than the displayed value, then the assumptions of Sec. 3.18 apply, 
the limit of error becomes ±0.0005 and the probable error becomes 0.00025. 

Thus, a truncated digital display of a measured quantity does not represent the most probable value 
of that quantity, but the most probable value can be obtained by adding a systematic correction equal 
to one half of the least significant digit. 

3.26 Average rate of random events , by counting. If a number n of discrete events, randomly separated 
in time, have all been counted in a time interval T (e.g., with a scaler), the computed frequency of occurrence 

3.26-1) ri 0 = n/T 

is not necessarily the same as the frequency n lT that would have been computed if the counting time had 
been many times larger than T. However, n 0 is the most probable value of the frequency, if it is the only 
information available. The rms fractional error (« 0 ) i° knowledge of n 0 is 

(3.26-2) Erfirio) = 5rt 0 /n 0 = 1/V« . 

The expected distribution of readings is known as the Poisson distribution. This distribution is not a continuous 
function— it exists only for integer values of n. However, when n is moderately large, a smooth curve drawn 
through the ordinates is adequately approximated by a Gaussian distribution (Fig. 3. 18(a) and Eq. (3.18-1)), 
in which o is 1/Vn 0 and e is the individual error in n 0 . (The curve through the ordinates of the Poisson 
distribution is slightly asymmetrical about e = 0 and is necessarily truncated on the left, since n > 0. When 
n = 10, the average difference between the two curves is less than 1 percent of the maximum amplitude 
(at c = 0). ) 

These idealized conclusions must be modified to correct for the imperfections of the instruments used 
in the measurements, limitations of the counting technique, and complications that may characterize the 
events being counted (D, C, and A sources of measurement error, respectively, in Sec. 3.2): 

(a) Dead-time error. The idealized formula is predicated on the assumption that the counting instrument 
is capable of resolving successive events no matter how little separated they are in time. In practice, events 
counters have only a limited counting speed, characterized quantitatively by their dead time X, an interval 
so short that two or more events occurring within the interval will be counted as only one event. If X is 
small enough so that n 0 X < < 1, the most probable value of the total counts is no longer n, but rather 

(3.26-3) N = n(] + n 0 \) 


where ri 0 is still given by Eq. (3.26-1). However, the most probable counting frequency now becomes 
(3.26-4) N 0 = N/T 
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if T has been measured with negligible error. The rms fractional error in N 0 remains 
(3.26-5) E^Nq) = W 0 /N 0 = 1/Vn 

(b) Background error. The results of this section find frequent application in the counting of radioactive 
disintegrations. In such cases, an additional error arises from the presence of spurious counts, due to the 
background of cosmic or ambient terrestrial radiation. This background must be measured separately. Every 
measurement must be corrected for the dead time of the events counter. If 

N b = corrected number of background events in time T 
N b = N b /T 

N t = corrected number of total counts 
N t = N t /T 


then the most probable frequency of occurrence of the events of interest is 

(3.26-6) N 0 = N t - N b 

and the rms fractional error in N 0 is 

(3.26-7) E a0 (N 0 ) = [V ( N t + N b )]/(N t - N B ) 

Equations (3.26-6) and (3.26-7) replace Eqs. (3.26-4) and (3.26-5). 

3.27 Average rate of random events, by integration. If the rate of occurrence of randomly occurring 
events, as described in Sec. 3.26, has been determined by use of an integrating-type counting-rate meter 
(frequency meter. Note N3.27) rather than by use of a counter and timer, and if the ratemeter has a time 
constant t, then 

(a) if a single, instantaneous observation of the meter yields a frequency h, the rms fractional error 
of n is 

(3.27-1) E a \(n) = dn/n = 1/V(2 nr) 

(b) if the meter is observed continuously over a time interval 7 and the arithmetic time average of 
its indication is ho, the rms fractional error of no is 

(3.27-2) E a \ (h 0 ) = (1 + T/r)-'[[l + T/(27')]/(ii 0 7’)) 1/2 

(c) if m readings are taken of the ratemeter indication, at equally separated time intervals Tim , and 
the m readings have an arithmetic average ho, the rms fractional error in h 0 is 


(3.27-3a) 

= M/fl)/V (2h 0 r) 

where 


(3.27-3b) 

A = [l + (m — 1)(1 -r 2 )] 

(3.27-3c) 

B = 1 + (m - 1)(1 - r) 

(3.27-3d) 

r = exp (-77(mr)] 
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Equations (3.27-2) and (3.27-3) are based on the assumption that the ratemeter had already been connected 
for a sufficient time to ensure the subsidence of starting transients; a criterion for this condition is that 

(1) the ratemeter indication no longer shows any appreciable steady, monotonic drift, or 

(2) the ratemeter had been connected for a time at least as great as 

(3.27-4a) At = r In (2nr) 


or 

(3 .27— 4b) At = 47, 

whichever is smaller. 

The corrections for background and for dead time of the counting chamber (like a Geiger-Mueller tube), 
given in Sec. 3.26, still apply. 

3.28 Combination of errors . Single measurement of a function of several variables . Assume that a quantity 

Z is computed, or otherwise determined, by measurement of k variables wi,w 2 through the 

relationship 

(3.28-1) z = «*) 

where each variable Uj has a most probable value Ujq, a systematic error Auj, and a random error bu r 
Then, prior to application of systematic corrections, the most probable value of z is 

(3.28-2) Zo = ^ w io» w 20 * • •• > u ko)- 

The systematic error in zo is, by Eq. (3.9-2) 

(3.28-3) Azo = H*(dz/dUj)*AUj 

(All summations in this section are from j = 1 to j = k. ) Because it is systematic, this quantity may be 
applied as a correction. 

If, a priori , the random deviations from the mean are equally likely to be positive or negative, and if 
the variables u } are independent of each other, then 

(3.28-4) (6zo) 2 = 'E(dz/du j ) 2 *(6uj) 2 

(Two variables are said to be independent of each other if the value of each does not depend on the value 
of the other.) 

Equation (3.28-4) differs from Eq. (3.28-3) and its equivalent, Eq. (3.9-2), in two important respects: 

(i) In Eqs. (3.28-3) and (3.9-2), the variables, u Jt need not be independent; in Eq. (3.28-4) the 
variables w ; must be independent— no one may depend on another, nor may any two depend on 
some common parameter. 

(ii) Because of the ambiguity in the sign of random errors, the summation of Eq. (3.28-4) involves 
squares of terms; random error terms are always additive— there can be no cancellation of terms. 


As indicated in Sec. 3.9, each of the error sources, A through E, listed in Sec. 3.2, usually represents 
an independent variable. The relative magnitudes of the terms in the summation of Eq. (3.28-4) indicate 
the relative importance of the various error sources in their effect on the accuracy of z. 
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In Eq. (3.28-4), the errors may be of any type, provided they are all alike; the equation holds for either 
the probable, average, or rms errors of the means. If 6 m ; represents the probable error of Ujq, then 6 zo 
represents the probable error of Zo* 

Sometimes, the fractional error is more convenient and meaningful than the absolute error. The practical 
forms which Eqs. (3.28-3) and (3.28-4) take in specific instances are shown by the following examples. 
(Subscript zero has been omitted for brevity.) 

Example 3.28a. If z = + u 2 + m 3 4 ... , then 

A z = AW] + Am 2 + Am 3 -I- ... 

( 6 z ) 2 = ( 6 w, ) 2 + (6u 2 ) 2 + ( 6« 3 ) 2 + ... 

Example 3.28 b. If z - u { u 2 u 3 .,., then 

A z/z = (Ami/mj) 4 (Am 2 /W 2 ) + (Am 3 /w 3 ) + ... 

(bz/z) 2 = ( 6 m,/M |) 2 4 ( bu 2 /u 2 ) 2 4 (5m 3 /m 3 ) 2 4* ... 

Example 3.28c. If z ~ upupup ..., then 

Az/z = a, (Amj/mi) + a 2 (Au 2 /u 2 ) 4 a 3 (AM 3 /a 3 ) 4- ... 

(bz/z) 2 = a 2 (bu\/U]) 2 4 a\ (bu 2 /u 2 ) 2 4- a 3 (5M 3 /M 3 ) 2 + ... 

Example 3.28 d. If z — In u { 4- a 2 In u 2 4* a 3 In m 3 + ..., then 

Az = ai(AMi/M() 4- a 2 (AM 2 /M 2 ) + a 3 (Aa 3 /M 3 ) 4 ... 

(bz) 2 = a 2 (bu\/u\) 2 4 a 2 (bu 2 /u 2 ) 2 4 a 3 ( 6 w 3 /M 3 ) 2 4 ... 

Example 3.28c. Power P generated in a resistor R is computed from m measurements of current 
I. Then the most probable value of P is [{U)/m] 2 R. It is not [( LI 2 )/m]R . 

Example 3.28/. Power P generated in a resistor is determined by first making m measurements 
of current / and then making n measurements of voltage drop E. Then the most probable value of 
P is [(£/)/m]«[(££)/ii]. 

Example 3.28g. Power P generated in a resistor is determined by making m pairs of simultaneous 
measurements of current I and voltage drop E. Then the most probable value of P is [L(EI)]/m. 

Example 3.28/r. The relation between Nusselt, Prandtl, and Reynolds numbers (N, P, and R, 
respectively) for heat transfer between the central section of a long cylinder and a fluid flowing 
in a direction normal to the cylinder’s axis is given by 


jV = 0 9 5 R 10-31 +0.037 log R) 


if 0.1 < R < 200 000. (The formula is derived from Fig. 10-7 in Ref. 3-2.) When R = 10 5 , 
10-percent uncertainties in P and in R yield an 8-percent uncertainty in N. 


Example 3.28i. A total radiation pyrometer measures the flux emitted from a body, whose 
temperature is T and whose total emittance is e, in accordance with the formula 



* e = Boe(T 4 - T 4 0 ) 


where T 0 is the pyrometer's temperature, a is the Stefan-Boltzmann constant, and B is a calibration 
constant characterizing the geometric optics of the instrument and the installation. If T 0 < < T % a 
1 -percent uncertainty in T is produced by a 4-percent uncertainty in e. 

On the other hand, a monochromatic radiation pyrometer, which measures the radiant flux 
x # AA that is emitted from a body in a narrow wavelength interval AX centered around a 
wavelength X, obeys the equation 

$Vx*AX * ex P [-c 2 /(Xr)] 

where c 2 = 0.0144 A>m is Planck's second radiation constant and e x is the monochromatic 
emittance of the body. Then, a 1 -percent uncertainty in T is produced by 

(i) a 5-percent uncertainty in e x when XT ~ 0.003 K*m (the product of X and T that produces 
maximum spectral radiance) 

(ii) a 14-percent uncertainty in e x when XT = 0.001 K # m. Thus, operation at a shorter 
wavelength is advantageous, if there is still adequate radiometric sensitivity. 

Example 3.28/ In the range 0<r<40 °C, the effect of temperature t on the emf £ of a saturated 
Weston standard cell is given by 

E = 1 018 220 — A£ — 53. 91 x — 0. 7 lx 2 + O.OlOx 3 


where x = / — 28 °C, / is in °C,£ and A E are in microvolts, and A E is a systematic correction 
furnished by the manufacturer for each cell. 

In the same temperature range, the effect of temperature on the emf of an unsaturated Weston 
standard cell is given by 

E = (1 019 300 + A£)(l - 0. 00283y - 0.1 12y 2 + 10. 5y 3 ) 

where y = (r — 20 °C)/1000. (A £, a systematic correction for the particular cell, is so small that 
it may be neglected in computing the temperature correction.) 

Then, the emf E will be known to 0.01 percent if 6(A£) < < 100 and if 

(i) the saturated cell temperature is known to 2 °C when the cell’s temperature is near 28 °C 
(the usual operating temperature); 

(ii) the unsaturated cell temperature is known to lie between 0 and 40 °C. 


Formulas (3.28-3) and (3.28-4) are not complete in a region where z is a maximum or minimum with 
respect to one of the variables Uj. The formulas fail completely if z is a function of only one variable u 
and dz/du — 0 at the point where errors are to be computed. Under such circumstances, the quantity 

(dzfduj) 9 Auj 


must be replaced by 


(dz n /duj) •(Au j ) ri /n\ 


where n is the smallest positive integer for which the corresponding derivative is not zero. 
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Example 3.28/r. In a particular electrometric titration wherein the potential difference u between 
two electrodes is related to the amount z of electrolyte added, the relation between the two variables 
in the vicinity of the equivalence point ( a,b ) is found to be given by 

z — b = c(u — a ) 3 . 


Then, at u — a, 


dz/du = 0; drz/du. 2 = 0; d y z/du y — 6 c 

so that 

A z = c(Au) 3 ; ( bz ) 2 = c 2 (5u) b . 

Example 3.28/. The relation between area A of a variable area channel and the Mach number 
M at that area, for a gas of constant specific-heat ratio k, is given by 

A = M,/M)[(l + bM 2 )/(/> + l)] (b+1)/(2i) 

where Z? = (/: — 1 )/2 and A x is the area at which Af = 1. To determine how the Mach number 
might change when the downstream channel area expands from A x to A x + A A, we note that, at 
M = 1 , 


dA/dM = 0 

b l AldM l = (Ik - 9 )/(k 2 - 1) * G 


so that 


(AM) 2 = (2/G)(AA/A x ) . 

Example 3.28m. The relative resistivity p/p x of manganin wire used for precision resistors, in 
the range 15 < t < 50 *C, is given by 

P/P, = 1 + 1.335 x 10 -4 jc — 4.245 x 10 "V 

where x = r-20 °C, t is temperature in °C, and p, is the resistivity at 20 °C (0.00484 fi«m). 

The relative resistivity of manganin sheet used for ammeter shunts, designed to operate at about 
45 °C, is given, for the range 15 < t < 80 °C, by 

p/p 2 = 1 - 3.28 x 10~ 5 y - 2.048 x 10 "V 

where y = t - 45 *C, and p 2 is the resistivity at 45 °C (0.00388 0* m). Consequently, 

(i) temperature must be known to within 1.5 °C if the value of a precision resistor is to be 
known to 0.01 percent; 

(ii) temperature must be known to within 7 # C if the value of a shunt is to be known to 
0.1 percent. 

3.29 The central limit theorem . It has been found that, when errors from various sources compound 
as given in Eq. (3.28-4), and each source has a different error distribution (although one that is symmetrical 
about e = 0), the error distribution of the result, z , closely resembles the Gaussian. The fact that the 
combination of errors from several independent sources tends to have a Gaussian distribution has been termed 
the “central limit theorem.” 
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Figure 3.29.— Sums of uniform error distributions, to illustrate the central limit theorem. 


This theorem can be illustrated by the combination of errors from several sources, each of which has 
a uniform distribution. Suppose that u ] has a uniform error distribution over the range —A < e\ < A, where 
e t = 6u [f and that u 2 has a uniform error distribution over the range —B < e 2 < B, where e 2 s &u 2 , and 
B > A. These distributions are shown in Fig. 3.29(a) and (b). (The ordinates of all distributions have been 
chosen so that they all enclose the same area, corresponding to unit probability.) Then the error distribution 
of z = u\ + u 2 has the form shown in Fig. 3.29(d). If, in addition, w 3 has a uniform distribution over the 
range -C < e 3 < C, (Fig. 3.29(c)) where e 3 == Su 3 , then the error distribution of z = + u 2 + w 3 has 

the form shown in Fig. 3.29(e)— a form hardly distinguishable from the Gaussian curve with the same average 
deviation or rms error, which is shown by the dashed line [Note N3.29]. 

3.30 Several measurements of a function of one or more variables . If the measurement of z described 

in Sec. 3.28 is repeated on several different occasions, leading to mean values Zq\,Zo 2 >... >Zot Zo m > each 

with its respective rms, average, or probable error feoi> 5 2 o 2 >-* *>^ 0 / &Zom> then the most probable value 

of z is its Grand Mean Zoo , which is the weighted average of the individual mean values, the weighting 
factor for each being the reciprocal of the square of the respective error (All summations are from / = 1 
to i = m) 


(3.30-1) zoo= p[zo ( /(5z6,]]/p[l/(5zo,) 2 ])- 

By Eqs. (3.19-18) and (3.19-19), the rms error ^(zoo) of the Grand Mean is given by 

(3.30-2) eloizoo) = ^,/E[1/(5zo,) 2 ] 

where 

(3.30-3) e 2 a , = [1 !{m - 1)] Ef^/^) 2 ] 

(3.30-4) v 0/ = Zoi — ^oo 

and 6zoi the rms error of zor 

If, instead, 6zo, is chosen to represent the average deviation e ai {zoi) or the probable error e pi (zoi) of Zqi> 
then Eqs. (3.19-16) through (3.19-21) provide the means for comparable expressions of c^(zoo) or 

^pO ( ^00 ) * 

The rms error given by Eq. (3.30-2) is said to represent the external consistency of the measurements. 
It should be comparable in magnitude to the rms value 

(3.30-5) [E[l/(& 0 ,) 2 ])“ l/2 


if there were no constant or systematic errors in any of the determinations z®. The expression (3.30-5) 
is said to represent the internal consistency of the measurements. 
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If e o0 in Eq. (3.30-2) is many times larger than expression (3.30-5), there is reason to suspect that the 
experiments that yielded the Zq, differed from each other by the presence of one or more systematic errors 
in one or more of the experiments. (This criterion is known as Gauss’s criterion.) Another phrasing is to 
say that if the differences among the means are much larger than the rms errors of the means, systematic 
errors may have gone undetected. Ordinarily, if e\\ as given by Eq. (3.30-3) is greater than about 10, 
there is reason to suspect the presence of a systematic error in some of the determinations of Zo- 

3.31 Indirect measurements of several quantities . If m quantities X\,x 2 ,...,Xj t ... **m cannot be measured 

directly, but if n functions f(x { ,x 2 ,...,Xj f ... t x m ) yield n measurements Sj (/ = 1,2 n) t then the most 

probable values of the m quantities can be obtained by minimizing the sum of the appropriately weighted 
squares of the differences between each pair of f and $ ( . If the weight of each measurement is w n it is 
required that 

n 

(3.31-la) 5 = ^ jc m ) — Si] 2 Wj = a minimum. 

i=l 

Then the most probable values of the jc’s are obtained by solution of the m simultaneous equations 

(3.31-lb) <pj m dS/dxj = 0 0 = 1,2,... ,m). 

This group of equations is termed the “set of normal equations.” 

The set of normal equations is generally difficult to solve unless the functions f are linear algebraic 
functions of the x’s. If they are not, it may be possible to find some function X of each x (like the logarithm, 
sine, or square) that does allow expression of the normal equations as a set of linear algebraic equations 
in the X's. 

If each / is a linear function of the jc’s, 

m 

(3.31-2) / = £ (otjjXj) 

j= 1 

one may readily compute not only the most probable value of Xj but also the uncertainty e a (x j) in that value, 
assuming that the distribution of errors is Gaussian. One arranges the equations <t>j = 0 in rows representing 
increasing values of y, and rearranges the equations so that they assume the form 

(3.31-3) a u x x + a l2 x 2 + ... + a tJ Xj + ... + a lm x m = y, 

a 2 ]X 1 + 022*2 + •• + <* 2 jXj + ... + 02*i*m = J 2 


0 , 1*1 + 0 / 2*2 + ••• + + ... + a jm x m = y } 


0 mi*i “b a m 2*2 + ... + a m jXj + ... + a mm x m — y m 

The solution of this set of m simultaneous equations in m unknowns yields the m quantites Xj. Denote 
the determinant of the coefficients on the left side of Eqs. (3.31-3) as D, where 


48 



§3.31 


tfll 

a n .. 

^\m 

a 2\ 

a 22 •• 

&2m 


(3.31-4) D = 


a m 1 a m2 • * ■ a mm 


and denote the cofactor of any diagonal term ajj as Then the rms error e 0 of x } is given by 

(3.31-5) el(xj) = ( Djj/D)*e 2 al 

where 

n m 

(3.31 -6a) (« - m)eh = £ x Pi 

i = 1 7=1 

or 

n 

(3.31-6b) (n-m)e 2 0 , = £ *m) ~ •s,] 2 *'; 

i=i 

Equations (3.31 -6a) and (3.31 -6b) are alternate expressions of the same quantity. The advantage of the 
first expression is that the two summations on the right side can be computed at the time that the original 
data are entered into any computer program; the disadvantage is that the right side represents a very small 
difference between two relatively large quantities so that each of these quantities must be computed with 
very high arithmetic accuracy. The advantage of the second expression is that it represents the direct summation 
of very small quantities, so that high computational accuracy is not required; the disadvantage is that the 
original data must be reentered into a computer program after the simultaneous equations have been solved. 


Example 3 . 31 . To calibrate a set of l-,2-,3-, and 5-gram chemical-balance weights, there are 
available a 10-gram standard and a lower-quality weight of nominal 1-gram mass. Denote the masses 
of these weights, respectively, as M [t M 2 , A/ 3 , A/ 5 , M \ 0 , and Af j. Calibrations are effected by direct 
measurement of the differences between nominally equal combinations of weights. The following 
measurements are made: 


(3.31— 7a) 

3/5 + A /3 + M 2 — A / 10 — 

(3. 31 -7b) 

A /5 — (A /3 + M 2 ) = 52 

(3.31-7c) 

A /3 - (A / 2 + M\) =53 

(3.31— 7d) 

js 

1 

as 

11 

(3.31-7e) 

Mi - (M 2 + A/ 1 ) =s 5 

(3.31-7f) 

M s - (Mi + A/,) + M\ =i 6 

Let 



49 



§3.31-3.32 


Each measurement s, is, a priori , assumed to have equal uncertainty 6s and hence equal weight w r 
In this example, n — 6 and m = 5 (because M\ is included), with x it x 2 , x 3 , jc 4 , x 5i corrsponding 
to M 5 , A/ 3 , M 2 , A/j, Af|, respectively. 

Application of Eq. (3.31-1) leads to the set of normal equations 



3A/ 5 - Mi 

— 


-M 5 + 5Mi 


(3.31-8) 


4Af 2 + 


-Mi 

+ M 2 + 


-M s 

+ m 2 


M\ — M j — Af jo + 5) 4- s 2 ■+■ 5^ 

= M\q + 5, - 5 2 -h53 + 5 5 -5 6 
A/j + A/ j = A/ iq 4 5j — s 2 — 5 3 — 55. 

3Af ] = 5 4 — 5 3 — 5^, 

+ 3Af , = - (j 4 -f5 5 + 5 6 ) 


which may be solved for M h M 2 , Af 3 , A/ 5 , and, incidentally, for M\. 

The value of D is 300. The diagonal terms of D are a u = 3, = 5, a 33 = 4, a u = 3, a 55 = 3, and 

their respective cofactors are Z) n = 150, D 22 = 66, D 33 = 96, D44 = 134, D 55 = 134. Consequently, 
e 2 c (M 5 ) = (l/2)e 2 aU e^(A/ 3 ) =0.224,, e 2 a (M 2 ) =0.324,, 4(A#i) = 4(^i) = (134/300)eJ„ 
where given by Eq. (3.31-6). 

In particular, if the values of Sj and of s 2 through s 6 are, respectively, —0.0025, -0.0015, 
—0.0075, —0.0035, —0.0115, and —0.0045, then e\\ = 8x 10“ 8 and the respective values of y Jt 
M p and e a (Mj) are given by the following table: 


y 

9.9915 

9.9845 

10.0180 

0.0085 

0.0195 

M 

4.9980 

2.9965 

2.0030 

1.00116 

(1.00483) 

e„(M) 

0.0020 

0.0001 

0.0002 

0.00019 

(0.00019) 


where the entries in the last column refer to the weight M j. (An overbar on a digit indicates that 
the digit repeats indefinitely.) 

3.32 Application to an experimental measurement The diagram of Fig. 3.32 shows how the results of 
this chapter may be applied to the measurement of a single physical variable. The numbers of the equations 
pertinent to the various steps are indicated. The words “probable error” may be replaced by “rms error” 
or “average error.” The steps of the measurement operation are 

1. Instruments having known probable errors are used to measure the experimental variables. 

2 . The measurement of each experimental variable is repeated n times to reduce the probable error by 
a factor of Vn. 

3 . The mean value of each experimental variable, as computed, is inserted in a formula or other known 
relation that gives the value of the final physical variable in terms of the measured experimental variables. 

4. The probable error of the final physical variable is deduced from the probable errors of the measured 
experimental variables. 

5. If steps 1 through 4 have also been performed by other laboratories or on other occasions, the means 
deduced by each of the laboratories are weighted in accordance with the probable errors of each laboratory’s 
results and then averaged to produce the Grand Mean. 

6. The probable error of the Grand Mean is computed from the results which were used to derive it. 
This provides a measure of the external consistency of the data. 

7. The probable error of the Grand Mean is computed on the assumption that only random instrument 
errors contributed to the Grand Mean. This provides a measure of the internal consistency of the data. 

8. The errors obtained in steps 6 and 7 are compared. If the first is much larger the the second, there 
is reason to suspect that an unrecognized systematic error entered into the determinations by some of the 
laboratories. 
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STEP 

1 

2 


3 


4 


5 


5 

6 

8 

7 


EQUATION 


3.13-1 


3.28-1 


3.28-4 


3.28-1 


3.28-4 


3.30- 1 

3.30- 2 


3.30-5 


Figure 3.32.— Sequence of experimental measurements. 


3.33 Calibration correction curves. A calibration correction curve is a graphical description of the 
systematic corrections to be applied to an instrument's indication. Such a curve for at least the first few 
units of a new instrument provides an understanding of the instrument's characteristics that may serve to 
determine the extent and nature of calibrations that should be made of subsequent units of the same type 
of instrument. 

Most commonly, the curve takes the form of a plot of Ay versus y in , where Ay is the correction to be 
added to the indication y in in order to yield the true value y tr of the quantity being measured. Only a small 
number of the pairs (y in , Ay) are usually determined; it is then necessary to decide how these points are 
to be joined in order to form a continuous curve. This decision usually cannot be made solely by inspection 
of the data; some knowledge of the nature of the instrument and of the calibration technique may also be 
required. 

(1) If it is known that exactly the same value of Ay will be obtained each time that the instrument 
is subjected to the input y tr , then the calibration curve may be drawn through each point (y in , Ay). Such 
a situation may exist if 

(a) the instrument and the determination possess negligible random error, or 

(b) Ay represents the mean of so large a number of observations that the rms error of the mean is 
negligible. 

Example 3.33a. It is known that the manufacturer of a presumably linear 100-division instrument 
scale actually calibrates and marks the scale at eleven cardinal points and then machine divides 
each interval into ten subdivisions. Then if random errors are negligible, a calibration may be 
performed at the cardinal points, and then these points may be joined by straight-line segments. 
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Example 3.33 b. It is known that the manufacturer of a digital millivoltmeter for a thermocouple- 
type pyrometer actually fits the thermocouple’s temperature-emf curve with a number of straight- 
line segments. If the end points of these segments are known, calibrations at these same end points 
may be joined by straight-line segments. 

(2) If it is known that the determination of Ay is subject to appreciable random error (due to the 
instrument or to the calibration technique), then a smooth curve should be faired through the points (y in , Ay). 

Example 3.33c. If, in Example 3.33b, the end points of the segments are not known to the 
calibrator, and if the ability to match the temperature-emf curve is considered to be a possible 
source of significant error, then a smooth curve should be faired through the points. 


Often, a straight-edge or French curve is sufficient to provide the desired calibration curve. If additional 
analysis of the curve is desired, the graphical and analytical techniques treated in the next chapter may 
be helpful; the analytical techniques are often needed if the data are to be entered into a computer program. 
The choice of a suitable technique, however, requires a human interpretation of the appearance of the 
calibration curve, based on an understanding of the physical operations involved. 

3.34 The designation of procedures in the specification of accuracy. Many legal contracts for materials 
or performance presume a measurement accuracy that is higher than is practically achievable. The resulting 
difficulties may be minimized by agreement on all details of the apparatus and techniques that will be used 
in the measurement, and agreement to accept the resultant indication as “true.” Such details would include 
the determination of the partial derivatives in Eq. (3.9-2) and the application of the resulting systematic 
corrections. The derivatives need be determined only once or only occasionally if similar sets of measurements 
will be made repeatedly over a long period of time. 

Many “standards” of national and international organizations are concerned with establishing 

(1) standardized conditions and procedures for the measurement of a physical variable, and 

(2) standardized methods for correcting for deviations from the standardized conditions (e.g., deviations 
from standard pressure and temperature). 

The resultant measurement represents “the value of the physical variable under the conditions of the 
specified standard.” 

The word “standard” may be replaced by the more precise term “recommended practice”. On some 
occasions, this substitution has some legal advantage because it avoids the interpretation that the “standard” 
is mandatory. 

3.35 Relation of accuracy to other factors affecting the selection of an instrument . Measurement accuracy 
is only one of many factors that affect the selection of a measurement technique and instrument. Some other 
factors are speed, reliability, convenience, and cost. Accuracy often drops when speed of measurement 
must be increased. Reliability of an instrument often increases as simplicity of instrument construction 
increases, even though such simplicity may involve some reduction in accuracy. Reduction of maintenance 
requirements, ease of repair and adjustment, and reduction of skills required for operation all affect reliability 
and overall cost; they may be related to the measurement accuracy that is obtainable. In some instances,the 
long-term availability of repair parts and services are also important considerations. The compromise among 
these factors depends on the weight assigned to each in any particular situation and depends on human 
judgment. Any formal mathematical procedures presented in this chapter are merely aids in the formulation 
of that judgment. Many of these procedures must be used with an appreciation of their approximate character, 
because they are based on mathematical assumptions chosen for their convenience rather than for their 
exactness. 
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NOTES FOR CHAPTER 3 

N3.9. An even closer approximation is possible if the experiment is repeated using an additional change 
(- Ax ,); or using a larger change 

(i) If the points on each side of are (x,- - Ax,, v, - Av, ) and (x, + Ax it 37 + A vs ) , then 


(dF/dx), = (Av, + Av 2 )/(2*Ajc / ) 


(ii) If the points on each side of (*,,>7) are (jc, - Ajc„ >7 - Ay,) and (x, + n*Ax lf >7 + Ay 2 ) , then 


( dF/dx) l = (rr»Ay, 4 Ay 2 )/[n(n + 1)*AjcJ 

(This combination of points is useful at the extreme ends of a tabulation.) 

N3.13 The molecular speeds depend only on the relative molar mass M and the gas temperature T \ If /? 0 
is the universal gas constant (8314.34 J*kmol _ ‘•K _1 ) then the arithmetic mean speed v fl and the rms speed 
v ff are given by 


v 2 a = SR 0 T/ ( tM); \ 2 a = IRqT/M. 

N3.18 If the number n of observations is small, so that assumption G for the Gaussian distribution does 
not hold, the distribution assumes a form resembling that shown in Fig. 3. 18(a) except that the peak amplitude 
at e = 0 is reduced, and the amplitudes at values of ela larger than about 1.6 are increased. This modified 
distribution is termed a t-distribution. Its distribution function is 

(N3. 18-la) /(/) =/ 0 *[l +t 2 /(n - 1 )] ~ n/2 

where 

(N3. 18-lb) / = e/[(Ev, 2 )/«] l/2 

(N3.18-lc) / 0 = [(« - 1 )ir] _1/2 r (n/2)/r[(n - l)/2] 

and v is given by Eq. (3. 14-1). 4 

Equation (3.18-2) is replaced by a proportionality that depends on n: 

(N3.18-2) e p : e a : e a = (0.65 to 0.67):(0.93 to 0.80): 1 
when n ranges from 4 to 32, respectively, and 
(N3. 18-3) e 2 a = (2n - l)/(2n - 3) 


4 In terms of factorials, the gamma function is given by 
T{n/2) = [(n - 2)!Vt]/[2"- 2 [fn-3)/2]!J if n is odd 

Y(n/2) = [(n — 2)/2]l if n is even 
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Table 3 . 18.2 lists values of t, for several values of n, at which the probability P of exceeding / is 2 , 3 , 
5, 10 , 15 , or 20 percent. When n > 4 and P > 5 %, the listed value of e corresponding to a particular 
probability does not exceed the value listed for a Gaussian distribution, by more than the uncertainty in 
the measure of dispersion, as treated in Sec. 3 . 22 . Hence, for n > 4, P > 5 %, use of the Gaussian distribution 
is adequate for all practical purposes. 

N3.19. The distinction between the deviation of a single observation, which serves to describe past events, 
and the deviation of the mean, which serves to estimate future events, is illustrated by the following examples, 
in which the error distribution is neither Gaussian nor uniform. 

(a) Suppose that n observations are made of a quantity y which is likely to have constant error by 
and that half of the observations yield a value y 0 + e and half of the observations yield a value y 0 — e, 
these two values occurring randomly. Then, if n is large, one may conclude that 

(i) The average value of y is y 0 

(ii) The average of the absolute value of the individual deviations is e, and this average deviation 
is substantially independent of the number n of observations 

(iii) In the future, if there is no change in the circumstances of the experiment, one may expect that 
the true value of y will be y 0 with an uncertainty that decreases as n increases. (The 
uncertainty will be on the order of e/Vn. ) 

(b) If the quantity y had been likely to have constant fractional error by/y and if the observations 
had yielded values of y 0 (l + e) and y 0 ( 1 — t) in equal number, then, in similar manner, one would conclude 
that 

(i) The average value of y is y 0 

(ii) The average fractional deviation of past observations is €, substantially independently of n 

(iii) The fractional uncertainty in assuming y 0 be the value of y in the future is about e/Vn 

N3.27 A frequency meter is an instrument whose indication y is almost proportional to the time integral 
of its input Y. Its behavior is described by 

y = (l/r)\(Y - y)dt 

where r is the characteristic time constant of the instrument and the integration interval is presumed to be 
much longer than r. This instrument is considered in Sec. 6.8. 

N3.29 I. When there are two uniform distributions 

-A < e A < A and — B < e B < B (B > A > 0) 


and 



e-\ e A + e B \; 

/= dP/de 

/= 1/(25) 

for 

0 < e < (B - A) 

f= (A + B- 

e)/(4AB) for 

(B - A) < e < (B + A) 
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II. When there are three uniform distributions 

-A < e A < A; -B<e B <B; -C<e c <C; (C > B > A > 0) 
and 

e - \e A + e B A- e c \; f = dP/de 

then 

For 0 < e < (B + A - C ) , 

/= 1/(2 C) if C> (B + A) 

/ = [4 AB - (A + B - C) 2 - e 2 ]/(&ABC) if C < (B + A) 

For (B + A — C) <e< (C - B + A), 

/= [8 AB - (A + B - C + e) 2 J/(16 ABC) 

For (C - B + A) < e < (C + £ — A), 

/= (5+ C- f)/(4fiC) 

For (C + B — A) <e< (C+B + /1), 

/ = (C+ B + A - e) 2 /(16 ABC) 

III. In the special case when A = B = C for all three uniform distributions. 

For 0 <<?</!, f=(3A 2 -e 2 )/(8A 3 ) 

For A < e < 3/1, /= (3/1 - e) 2 /( \6A 2 ) 

and e p = 0.1XA, e a - 0.8M, e 0 - A. 
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CHAPTER 4. EMPIRICAL GRAPHS AND EQUATIONS 


4.0 Introduction . Symbols. The material in this chapter is relevant principally to the interpretation and 
manipulation of experimental data, in order to improve the understanding of what is being studied, to plan 
the course of future operations and to monitor their progress, to analyze and to present results, and to present 
conclusions. 

The following symbols are used frequently in this chapter. 

e a or a average error or deviation 
E a fractional average error or deviation 

e a or o rms error (standard deviation) 

E a fractional rms error (fractional standard deviation) 

m number of independent constants 

n number of observations 

r correlation coefficient 

S sum of squares of residuals 

V fractional residual 

v residual 

w weight of observation or of square of residual 

x independent variable 

y dependent variable 

p x >p y index of correlation 

Symbols used occasionally will be defined as they are introduced. 

4.1 The uses of empirical graphs and equations. The complete understanding of an instrument's 
characteristics may require the determination of how the instrument is affected by several variables other 
than the principal variable that the instrument is to measure. Similarly, the complete understanding of a 
measurement technique may require the determination of the influence of each of several variables that 
influence the measurement. Quantitatively, these determinations may involve the determination of the form 
and magnitude of the partial derivatives that appeared in Eqs. (3.9-2) and (3.28-3). 

Example 4.1 a. The natural frequency of a tuning fork or crystal may be affected by its orientation 
and by the pressure, temperature, and composition of the medium in which it is immersed. 

Example 4.1 b. The indication of a thermometer bulb immersed in a flowing fluid is affected by 
the Reynolds number of the flow, the Prandtl number of the fluid, and the shape and physical properties 
of the bulb’s support. 


When a new experimental procedure is initiated, or a new type of instrument is tested, initial experiments 
seek to determine the systematic effects of one variable at a time. Dimensional analysis is a powerful tool 
in minimizing the number of independent variables that need to be identified. Initially, graphical methods 
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are effective in clarifying the nature of the relationships between dependent and independent variables. Once 
this identification has been made, analytical methods of describing the relationships may be used 

(i) to implement subsequent mechanized operations, like calibration, or 

(ii) to facilitate subsequent routine data analysis, or 

(iii) to improve understanding of the instrument and of the measurement technique and, thereby, to 
improve the operation to which they are applied. 

Graphical methods help to monitor an experiment by identifying anomalies in the data or in the procedures 
(or, conversely, by confirming their reliability). Graphs, and equations that represent them, are also compact, 
easily comprehended means of presenting results. In this chapter, such empirical graphs and equations will 
be considered. 

When the relationship between two variables x and y is determined by a few pairs of simultaneous 
measurements (jc |f y/) over a range X\ < jc < Jt m , and if y(jc) is continuous over this range, a graph or an 
analytical equation, that represents the most probable relationship between the two variables, offers two 
advantages; 

(i) It provides greater accuracy than does the mere tabulation of the values themselves, because 
all the data jointly contribute to the result. 

(ii) It permits interpolation between the pairs of values that represent the actual measurements. 

The operation of fairing a smooth curve through a set of discrete points is, thus, analogous to taking 
the mean of a set of measurements of a single quantity or, more precisely, to determining the most probable 
value of a function of several variables. 

4.2 Graphs . There is no mathematical law that will determine what type of curve should be drawn through 
a set of plotted points. The decision whether the curve should be a straight line, or a parabola, or whether 
it should have inflection points (and, if so, how many) must be made by the analyst on the basis not only 
of the appearance of the plotted data, but also of the analyst's understanding of the physical phenomena 
represented by the data. The decision becomes less difficult and more obvious as the number of data increases 
and as the scatter of data from an assumed curve decreases. Conversely, as the physics of the situation 
is better understood, fewer pairs of measurements are needed to establish the faired curve. The criteria 
used in fairing are also optional; usual human judgment is to minimize the sum of the absolute value of 
the distances of the plotted points from the faired curve. If rectangular Cartesian coordinates are used, these 
distances may be measured in a vertical direction, in a horizontal direction, or in a direction normal to 
the curve. 

Once the form of the curve has been chosen, it may be convenient to represent it by an analytical formula. 
However, the criterion used to select the original graph may no longer be convenient for an analytical 
treatment; usually, considerations of mathematical convenience dictate that, to deduce an analytic formula, 
the least-squares criterion should be used and that it should be applied to only one of the variables. This 
subject will be treated more completely in later sections. 

4.3 Equations . The operation of finding an analytical expression for the relationship between jc and y 
is actually the operation of determining the most probable values of the constants of an assumed empirical 
equation. There is no mathematical procedure that will tell in advance what form of equation is followed 
by a given set of data. [Note N4.3. 1] Mathematics can serve only to tell how closely an equation of assumed 
form fits the actual data. The selection of the form of the equation to be assumed requires human judgment; 
that judgment is made from a mental comparison of the faired curve with the graphs of known analytic 
functions, from a physical intuition concerning the relationship between jc and y, or by trial and error. The 
judgment is also influenced by whether the range of x that is of interest does or does not extend beyond 
the range of the data. 

Once the form of the equation has been selected, presumably for use with future sets of experimental 
data, the criterion for selecting the best fit (i.e., the most probable values of the constants in the equation) 
must be specified. The least-squares criterion is the most common choice because of its convenience— one 



minimizes the sum of the squares of the deviations of the measurements from the value predicted by the 
equation. Another criterion is the Chebyshev criterion— one minimizes the maximum deviation of the 
measurements from the value predicted by the equation. For any criterion, the range of the variables over 
which the criterion is to be applied must be specified. An equation which has been computed to provide 
the best fit over a specified range may not generally be used to extrapolate beyond that range [Note N4.3.2]. 

After the best-fit criterion has been chosen, some additional decisions must be made before the formal 
equation-fitting operation can be started. These decisions, discussed in Sec. 3.15, require assumptions of 
the relative weights to be assigned to the various experimental observations. 

The trial selection of the form of equation to be fitted to a set of data is often made by inspection of 
the plot ofy(.r) and by the pragmatic consideration of expediency of mathematical operations. For example, 

1. if y(x) is always positive and symmetrical about the ordinate x = x 0 , then y(x) should contain 
only even powers of (x - jc 0 ). 

2. if y(x) lies only in the first quadrant, is monotonic, and is concave upward, a formula of the form 

(4.3-1) y = La i x‘ (i = 0,1.2,...) 

may be appropriate. 

3. if y(x) lies only in the first quadrant, is monotonic, and is concave downward, a formula of the form 

(4.3-2) y = Ea,(Vjr)' (i = 0,1,2, .. .) 

also may be appropriate. 

4. if y(;t) lies in the first and third quadrants, is symmetrical about the origin, and is monotonic, 
an expression like Eq. (4.3-1), with only odd values of /, may be appropriate. 

5. if y(x) lies in the first quadrant, is monotonic, and appears to approach a horizontal asymptote, then 

(4.3-3) y = ^ / (* + E biX ‘ 

is a general form that may be appropriate. 

6. if y(x) has a point of inflection, any assumed polynomial must be at least of the third degree. 

7. if y(0) = 0, then a 0 should be zero in Eqs. (4.3-1) to (4.3-3). 

4.4 The case of both variables subject to error . When both variables, x and y, are subject to error, the 
fitting of an appropriate graph poses no serious problem; a convenient criterion is to minimize the sum 
of the visually measured distances of the plotted points from the faired curve. 

However, when the curve fitting is to be performed analytically, by means of some chosen form of equation, 
the required mathematical operations are usually complex and inconvenient, and require an advance estimate 
of the rms errors of each of the variables. It is more expedient to assume that only one of the variables 
is subject to error and that the measurements of the other variable are exact. This latter variable is chosen 
as the one which is believed to have the smaller fractional rms error. In this text, x will designate the variable 
which is assumed to be without error. 

An important exception occurs when there is a linear relationship between x and y. It then remains convenient 
to treat both x and y as being subject to error and to describe mathematically the correlation between the 
two variables. Correlation is treated in Sec. 4.12. 

4.5 Criteria for best fit of a curve or equation . The fitting of a curve or equation to a set of measured 
points (*,,y,) depends on what prior assumption is made concerning the relative reliability, or weight , of 
the individual points and, as a practical matter, on whether graphical or analytical methods are chosen. 

In fairing a curve through plotted points, one seeks to minimize the sum of the visually measured distances 
of the points from the curve, treating all distances as positive. The operation is analogous to taking the 
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arithmetic mean of a set of data. The nature of the coordinate system and the relative fractional accuracy 
of x and y determine whether the distances will be measured in the direction of x, or of y, or of the normal 
to the curve at that point. For example, if rectangular Cartesian coordinates are used, then 

(a) If the a priori uncertainties in jc and y are represented by distances of comparable magnitude, 
deviations should be measured along the normals to the curve. 

(b) If the a priori uncertainty in y is represented by a much larger distance than is the uncertainty 
in jc, deviations should be measured in the direction of y. 

A graphical construction is particularly advantageous when both jc and y may have appreciable errors. 
In fitting an equation to a set of points, mathematical convenience requires that jc be presumed to be without 
error. One then seeks to minimize the sum of the appropriately weighted squares of the differences between 
the measured value and the value predicted by the equation. These differences will be termed residuals , 
as in Sec. 3.14, so that if the fitted equation is v = /(jc), then, for any point (jc,,y,), 

(4.5-1) v, =/(*■) -y h 

Case 1. If each y t is presumed to have substantially the same absolute uncertainty (5y = constant), then 
we seek / such that 

(4.5-2) £v 2 = a minimum. 

(All summations not otherwise identified are over the index /. ) 

Case 2. If each observation y, is presumed to have the weight w,, then we seek / such that 

(4.5-3) Evfa = a minimum. 

Case 3. If each observation y, is presumed to have the same fractional uncertainty (by/y = constant), then 
the weight of each observation is 1/y 2 so that we seek / such that 

(4.5-4) £v 2 (1/y 2 ) = a minimum. 

In an alternate phrasing, we could have defined a relative residual 

(4.5-5) V, = [f( Xi ) - y,]/y, 

and sought / such that 

(4.5-6) = a minimum. 

Equations (4.5-1) and (4.5-5) are comparable to Eqs. (3.14-1) and (3.19-8). 

If there are n pairs of data (i = l,2,...,n) and if /(jc) has m constants whose most probable values are 
to be determined, then a mathematical measure of how well /(jc) agrees with the measured points is given 


by € a \ or where 


(4.5-7a) 

(n-m)eh = £v 2 

for Case 1 

(4.5-7b) 

(n-m)e 2 a] = Ev?w, 

for Case 2 

(4.5-7c) 

(n-m)E 2 a \ = £v? 

for Case 3 


= E(v?/y?) 

for Case 3 
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Note that the dimensions of e aX are different in the three cases. In Case 7, Eq. (4.5-7a), the dimensions 
of e a \ are the dimensions of y ; in Case 2, Eq. (4.5-7b), the dimensions of e a] are the dimensions of y V w; 
in Case 5, Eq, (4.5-7c), E a[ is dimensionless. 

In the case of graphs, the definition of Eq. (4.5-1) is usable when uniformly spaced rectangular coordinates 
are used; then, if all v, are presumed to have the same absolute uncertainty, the nature of human perception 
requires that the criterion (4.5-2) for Case 7 be replaced by 

(4.5-8) E|v,| = a minimum. 

When logarithmic coordinates are used, and all v, are presumed to have the same fractional uncertainty, 
the definition of Eq. (4.5-5) is usable and the criterion (4.5-6) for Case 3 must be replaced by 

(4.5-9) E|K,| - a minimum. 

Criteria (4.5-8) and (4.5-9) for graphs usually do not produce results that are significantly different than 
those that would be produced by the criteria (4.5-2 and 4.5-6) that they replace. 

Case 2 cannot usually be handled satisfactorily by graphical means if w, is neither constant (so that 
uniformly spaced rectangular coordinates may be used) nor proportional to 1 fy 2 (so that logarithmic 
coordinates may be used), unless special graph paper is employed. An exception is provided by the use 
of error bars or rectangles, as described in Sec. 4.7. 

Approximate formulas, comparable to Eq. (4.5-7), that provide a measure of how well the graph and 
the original data agree with each other, are 

(4.5- 10a) n*e aX = E|vJ 

(4.5-10b) n»E al =L |K f | 

where e aX and E a[ are termed average errors. 

There remains the question of how to choose among various forms of the function/, if an equation is 
to be fitted, or among various shapes of curves, if a graphical solution is sought. There is no rigorous 
mathematical answer to this question; the answer is subjective and depends on the analyst’s understanding 
of the physics of the phenomenon being studied and of the quality of the experiment. It is commonly felt 
that the simplest form of equation or curve should be sought. Gauss’s criterion was to choose that form 
of/ which yielded a minimum value of e oX or E a] for a given m. Ordinarily, e aX can be made smaller by 
making m larger. Obviously, in the extreme condition, e aX can be made zero if m is chosen equal to n ; 
the curve then passes through each datum, with the implication that each measurement is perfectly accurate. 
Some practical rules of thumb that have been used are that a larger value of m should not be chosen unless 
e a] or E oX is thereby reduced by a factor of at least 1 .5, and that m should not exceed nl 4 unless previous 
experience has demonstrated that / can be reliably defined by fewer than 4 m measurements (e.g., if/ is 
known to be a straight line). 


4 . 6 . Accuracy of a curve or equation . The quantities e oX or E aX in Eq. (4.5-7) are measures of how 
well the curve or equation agrees with the measured data already obtained. They have clear, simple meanings 
only when each measurement is deemed to have equal uncertainty (Case 7) or equal fractional uncertainty 
(Case 3). Their practical meaning may not be obvious if the measurements have other weights. 

However, in future use of the curve or equation to estimate the most probable value of y(x) for any 
given Jt, the uncertainty in y(jc) is given by e^iy) or (y), where 
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(4.6- la) 


for Case 1 


elo(y) = eh In 
(4.6-lb) elo(y ) = for Case 2 

(4.6- lc) Eloiy) = £ai In for Case 3 

The dimensions of e^iy) are always the dimensions of y; E^iy) remains dimensionless. The quantity 
(y) has a clear, simple meaning even for weighted observations of any type. 

Whereas e aX and E oX hardly change when n increases considerably, the measures e^(y) or E^iy) 
decrease in proportion to Vn in Cases 1 and J, or to V(Ew,) in Case 2. Thus, the accuracy of future 
estimated values of y(jt) increases appreciably with the number of data that were used to determine /( x) 
originally. 

By similar reasoning, the uncertainty in future estimated values of y by means of a graph is given by 
eao(y) or Eao(y) f where 

(4.6-2a) eaoiy) = e aX /yin 

(4.6-2b) Erfiy) = E aX /\In 

and where e aX and E aX are given by Eq. (4.5-10). 

4.7 Error bars and rectangles . The graphical construction of a curve through a set of points plotted on 
rectangular coordinates may be facilitated by constructing, around each point, a rectangle whose width and 
height represent the estimated probable, average, or rms uncertainties in x and y at that point. If only y 
has appreciable error, the rectangle is replaced by a vertical bar. The simplest curve that passes through 
all the rectangles or bars is likely to provide an adequate representation of the relation between x and y. 
The generalization to other coordinate systems, such as the polar, is apparent. The size of the error rectangle 
or bar varies inversely with the weight which that point should have in defining the most fitting curve. 

An error bar may be used for a different purpose in the graphical presentation of results obtained by 
graphical or analytical means. It may be erected at each measurement point, or merely at one point anywhere 
along the graph, to represent the value of e aX (y) or E aX (y) deduced from the operations treated in Sec. 
4.5. In this usage, the bar represents a conclusion rather than an a priori assumption. 

Example 4 . 7 . An oscillograph is used to record the decay of voltage E , from an initial value 
E 0 , when a capacitor C is discharged through a resistor R. The applicable physical law is 
E = Eq exp [— // (RC)]. The data are to be replotted by using the dimensionless variables y = E/E$ 
and x = t/{R x C x ), where R x and C x are the nominal values of R and C. The a priori uncertainty 
in E is 0.06 £ 0 . 

If the a priori uncertainty in x is negligible, and the data are plotted on uniformly spaced rectangular 
coordinates, the graph will appear as in Fig. 4.7(a), wherein each point is represented by a vertical 
error bar. If there is also an uncertainty in jc of magnitude 0.04., the graph will appear as in Fig. 
4.7(b), wherein each point is represented by an error rectangle. The advantage of the constructions 
of Fig. 4.7(a) and Fig. 4.7(b) is that each point has equal weight because each error bar or error 
rectangle is of the same size. 

If the uncertainty in x is 0.04, and the data are plotted on semilog paper, the graph will appear 
as in Fig. 4.7(c). The advantage of this construction is that it is known in advance that the graph 
should be a straight line. However, at small values of y. the vertical height of error bars or rectangles 
becomes large and also asymmetrical about the plotted point. 
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(a) <5x = 0, <5y = constant 

(b) 6x =s constant, 6y = constant 

(c) 6x = constant, 6y = constant 
Figure 4.7.— Uses of error bars 

and rectangles. 

4.8 Linearized graphical constructions. It is usually advantageous to deal with a linear relation between 
the independent and dependent variables, because then moving a straightedge about on a sheet of graph 
paper permits visual estimation of the most fitting straight line. If a linear relation does not exist, then 
transformation of variables may serve to produce a linear relationship. Such transformations must always 
be used with full consideration of the changes they make in the error rectangles or bars. 

To facilitate the use of linear relations between transformed variables, special graph papers are available. 
In addition to the common-logarithm scales, there are also papers which provide a scale of reciprocals, 
of the logarithm of the logarithm, or of Gaussian probability (\f(e)^de, where fie ) is given by 
Eq. (3.18-1)). 1 

A particular advantage of the use of such a special scale is that it provides a guide to the planning of 
experiments, it is desirable that data be taken at values of the independent variable that are equally spaced 
on the scale representing that variable. 2 

Some examples of the transformation of variables to yield a linear relationship are listed here: 

1 . If the variables can be arranged or transformed to produce a straight line on uniformly spaced 
rectangular coordinate paper, the y-axis intercept b and the slope m define the relation 

(4.8-1) y = mx 4- b. 


'For example, the kinematic viscosity v of lubricating oils as a function of temperature T often forms a straight line on graph paper 
whose coordinates are proportional to log T and to log log v. 

2 Some experimenters prefer to have slightly closer spacing near the ends of the range of measurement. 
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2. If the variables can be arranged or transformed to produce a straight line on semilog paper 
[Note N4.8] with uniformly spaced ^-coordinates, and if 

(a) the intercept on the log y axis is log B , 

(b) jcj and x 2 are the abscissa of two points for which y(x 2 ) = 10 # y(*i)> 

then 

(4.8-2a) y = B exp (x/M) 

where 

(4.8-2b) M = (jt 2 — jcj ) /In 10. 

For example, in Fig. 4.7(c), B = 1 and M = — 1. 

3. If the variables can be arranged or transformed to produce a straight line on log paper whose 
horizontal and vertical decades are of equal length, and if 

(a) the line passes through the point (log x ]f log y^ 

(the scale markings for this point are and y\) 

(b) the slope M of this line is taken as the ratio between the lengths (in cm or other unit of length) 
of the vertical and horizontal legs of a right triangle erected on the line with the line as hypothenuse, 

then 

(4.8-3) y = (y x /x?)*x M . 

4. If the intercept y 0 on the y-axis of rectangular Cartesian coordinates is clearly defined, and there 
is also a clearly defined horizontal asymptote y m , one of the transformations 

(4.8-4a) Y = {y-y 0 )/ (y m - y 0 ) 

or 

(4.8-4b) Y= (y - y m )/(y Q - y m ) 

may lead to a linear relationship between x and Y. 

5. If the intercept x 0 on the jc-axis is clearly defined, and there is also a clearly defined vertical 
asymptote one of the transformations 

(4.8-5a) X = (x -x 0 )/(x m - x 0 ) 

or 

(4.8-5b) X = (x - x m )/(xQ - x m ) 

may lead to a linear relationship between X and y. 

6. If y(x) on uniformly spaced rectangular coordinates is monotonic, has no inflections, and approaches 
a horizontal asymptote as x decreases, plot Y = 1/y versus x on uniformly spaced rectangular coordinates. 
If a straight line of the form 

Y=mx + b 
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results, then 

(4.8-6a) y - ax/{ 1 + cx) 

where 

(4.8-6b) a - 1 /b; c = ma. 

7. If y(x) on uniformly spaced rectangular coordinates passes through the origin, is monotonic, has 
no inflections, and approaches a horizontal asymptote as x increases, plot Y = 1 fy versus X = l/x on 
uniformly spaced rectangular coordinates. If a straight line of the form 

Y = mX + b 


results, then 

(4.8-7) y = x/(bx + m). 

8 . If a plot on semilog paper (with logarithmic y-scale) shows only a slight curvature, and there are 
two points (xi,yi) and (* 3 , > 3 ) near the extremities of the curve which have good accuracy, 

(a) compute^ = (x, +x 3 )/ 2 , 

(b) find y 2 = y(x 2 ) on the curve, 

(c) compute y 0 = ~ y\)/(y\ + ^3 ~ 2y 2 )- 

Then a plot of y - y 0 versus x on semilog paper may yield a straight line. If it does, follow example 2 
of this listing. 

9. If a plot on log paper shows only a slight curvature and there are two points Ui.yi) and (x 3 ,y 3 ) 
near the extremities of the curve which have good accuracy, 

(a) compute y 2 = ^(y^), 

(b) find x 2 — x(y 2 ) on the curve, 

(c) compute x 0 = (x^ - x 2 )/(x l + x 3 - 2x 2 ). 

Then a plot of y versus x — x 0 on log paper may yield a straight line; if it does, follow example 3 of this 
listing. 

Alternatively, 

(a) compute x 2 = V (xjx 3 ), 

(b) find y 2 = y(x 2 ) on the curve, 

(c) compute y 0 = (yitt ~ y\)/(y\ + >3 - 2> 2 ), 

and, if a plot of y — .y 0 versus x on log paper yields a straight line, follow example 3 of this listing. 

10. If y(x) plotted on uniformly divided rectangular coordinates passes through the origin and shows 
only moderate, monotonic deviation from the tangent at the origin, a plot of Y = y/x versus x may yield 
a straight line 


Y = Mx + B. 


Then 

(4.8-8) y = Bx + Mx 2 . 


4.9 Linear approximations of a quadratic. Measures of nonlinearity. Many nominally linear instruments 
(i.e., instruments whose indication is supposed to vary linearly with the quantity being measured) actually 
show a slight nonlinearity, so that the relation between the variables is more closely represented by a quadratic 
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TABLE 4.9 -PARAMETERS OF SIX METHODS OF DESCRIBING NONLINEARITY 


Parameter 


Method 



Zero-based 

Minimum error 


flj Hk 

B 

c 

D 

E 

F 

BpHlS 


(mm. LE) 

(min. e g ) 

(min. e„) 

(min. LE) 

(min. e a ) 

n 


1 +(12-8 V2)A 



IHHH 


I 


0 




■ 

1 





(V2-D/4 

M 

*2 

1/2 

V2 — 1 

3/8 

1/2 

1/2 

1/2 

*3 


2(V2-I) 

3/4 

(V3 + l)/6 

(y/2 + l)/4 

3/4 

|v(0)| 

0 

0 

0 

2A/3** 

A/2** 

3A/4** 

|v<Jr 2 )f 

A** 

( 12 - 8V2)A * 0.69A * * 

9A/16** 

A/3 

A/2** 

A/4 

|v(l)| 

0 

(12-8\/2)A«0.69A** 

A 

2 A/3** 

A/2** 

3 A/4* * 

*5 

8A2/15 

1 6(78 - S5V2)A 2 / 1 5 

A 2 /5 

8A 2 /90 

7A J /60 

23A 2 /240 


-0.73A 

-0.48A 

-0.45A 

• 0.30A 

-0.34A 

•0.31A 


2A/3 

4(37V2-S2)A/3 

19 A/4 8 

4V3A/27 

(2V2-DA/6 

A/4 


• 0.67A 

-0.43A 

** 0.40 A 

*0.26A 

*0.30A 

0.25A 


Limit of error (LE) 




x 2 x 3 t 



(a) Chord method 

(b) Zero-based 
methods 

(c) Minimum-error 
methods 

Figure 4.9.— Methods 
of describing 
nonlinearity. 


equation. In such cases, the manufacturer or the user may choose, for convenience, to approximate the 
relation by a straight line, and to provide some measure of the limit of error (or average error, or rms 
error) that may be expected by use of the approximation. The description of the error may be a statement 
like “nonlinearity does not exceed p percent of span. ” Some of the criteria that are commonly used to 

select the straight line approximation will be treated here, as well as the measures of nonlinearity that are 
associated with them. J 

It will be assumed that the relation between the variables is represented by a quadratic curve that passes 
through the points (A^,y 0 ) and (X m ,Y m ) at the extremes of the instrument’s range. To a user, it is desirable 
that X represent instrument indication and Y represent the physical variable that the indication is supposed 
to represent. However, a manufacturer may prefer to reverse the roles of X and Y. In any event, X will 
be treated here as the independent variable. For convenience of presentation, we introduce the transformations 

(4.9-la) x = (X — Xq) / (X m — Xq) 0 < jt < 1 

(4.9-lb) y= (Y-r 0 )/ ( y m -y o) O^sl 

so that the relation between x and y appears as shown in Fig. 4.9 and 
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(4.9-2) y(x) = jc + 4x( 1 - x)A 

where A is the difference, at abscissa x = 1/2, between y(jt) and the chord through the end points; A is 
also the maximum difference between the curve and the chord. 

The quadratic parabola of Eq. (4.9-2) may be approximated by a straight line whose general equation is 

(4.9-3) y = mx + b 

where the parameters m and b are chosen in accordance with some criteria selected by the instrument user. 
Each straight line leads to a residual 

(4.9-4) v(jc) = y(jc) — (nix + b) 

as the difference between y{x) in Eq. (4.9-2) and the chosen straight-line approximation, Eq. (4.9-3). 
Table 4.9, together with Fig. 4.9, shows six different approximations that may be used, the salient features 
of each, and the values of the various measures of nonlinearity that describe how well the approximation 
resembles the actual quadratic parabola. These measures of nonlinearity usually take one of the following 
forms: 

(a) A statement of the maximum value of |v|, the limit of error (LE), in the range of the instrument 

(b) A statement of the average value e a of |v| over the range of the instrument 

(c) A statement of the rms value e 0 of v over the range of the instrument. 

As indicated in Sec. 3.5, each of these measures can be expressed in units of y, or as a percentage of 
the instruments’ span or full-scale value. When the dimensionless representation of Eq. (4.9-1) is used, 
all three measures are necessarily dimensionless. 

The six approximations treated in Table 4.9 fall into one of three categories: 

(I) Approximation A, represented in Fig. 4.9(a), is a chord specified to pass through both end points. 
A specification that reads 4 4 the deviation of the actual calibration curve from a straight line through 
the end points shall not exceed 1 percent of the instrument’s span” is more difficult to meet than 
if the ”1 percent” had been used in any of the other five approximations. Consequently, this 
specification leads to a higher instrument cost. 

(II) Approximation D, E, or F, represented in Fig. 4.9(c) by a straight line parallel to the chord through 
the end points, but offset vertically by an amount that depends on the criterion chosen, places 
the fewest demands on the manufacturer, thereby minimizing costs, but requires that the user 
incorporate both parameters m and b into his mathematical analysis of the data. The actual curve 
and the approximation intersect at abscissae x { and where v is zero. Maximum positive and 
negative values of the residual occur at x = x 2 = 1/2, at x = 0, and at x = 1. 

(III) Approximations B and C, represented in Fig. 4.9(b), are each a straight line through the origin. 
They lie between categories I and II in the demands placed upon the manufacturer and the user. 
The actual curve and the approximation intersect at abscissa x = jc 3 , where v is zero. Maximum 
positive and negative values of the residual occur at x = 1 and at some intermediate abscissa 
jc = x 2 . 

4. 10 Analytic determination of the constants of an equation. The most probable values of the constants 
a ]v a 2 '-»''a,',... y a m of an assumed equation relating several variables can be determined by purely 

analytic means, if the criterion for defining the most probable values is defined. This criterion is usually 
taken as the requirement that the sum of the appropriately weighted squares of the residuals shall be a 
minimum; the procedure is then termed 4 ‘the method of least squares. ” Some other criteria were illustrated 
in Sec. 4.9, in application to an assumed linear relationship. When a nonlinear formula is assumed, application 
of these other criteria often becomes awkward, while use of the method of least squares remains mathematically 
convenient. This convenience is a reason for the method’s popularity. It will be used exclusively in the 
applications that will henceforth be treated. 

If a measured variable s is dependent on a number of independent variables x and a number of 
as yet undetermined constants aj (where j = l,2,...,m) so that 
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(4.10-1) 5 = F(x f y,z a l ,a 2 ....,a J ,...,a m ) 

and if the weight of each measurement 5/ of s is w if then the least-squares method is used to determine 
the values of the a } which make 

(4.10-2) 5 = £(F, — $i) 2 Wj = a minimum. 

Here 

(4.10-3) F f = F(x it y ir Zi ai,a 2 ,... f a jf ... 9 a m ) 

and the summation is for / = 1, 2, ..., n. It is necessary that 

n ^ m. 

(The equality sign represents the condition 5 = 0 because the number of undetermined constants equals 
the number of measurements and, hence, of equations.) 

The general method of solution, which parallels that of Sec. 3.31, is to create and to solve the set of 
m simultaneous equations 

(4.10-4a) <pj * dS/daj = 0 O' = 

which is equivalent to the set 

(4.10-4b) ^/F i m (dF/da J )iw i = I ^•(dF/daj) i w i . 

This set is termed the “set of normal equations”. 

In the following sections, this general method will be applied to some simple situations that permit explicit 
computation of the aj and of the associated measures of uncertainty. In particular, for each assumed form 
of the function F, three cases will usually be considered: 

Case A. The a priori uncertainty 6s in s is a constant. Then w, = 1 for all /, and it is necessary that 

(4.10-5) S A = E(F, — s,) 2 = a minimum. 

Case B. The a priori fractional uncertainty 6 s/s in s is a constant. Then the weight of each term in the 
summation (4.10-2) is l/$ 2 and it is necessary that 

(4.10-6) S B = £[(Ff/Sj) 2 - 1] = a minimum. 

Case C. The weight w, of each measurement s, is other than unity or 1 is}. Then it is necessary that 
(4.10-7) S c = ^(F, — Sj) 2 w t = a minimum. 

For each form of F, formulas for the following quantities will be given: 

1 . The most probable value of each independent aj 

2. The rms value e ^ or of the uncertainty (absolute, or fractional) that may be expected, in future, 
by use of the deduced a y assuming that the circumstances affecting future measurements are not 
different from those that governed the original measurements. The values of e ao and E ^ will 
usually be expressed in terms of an intermediate parameter e al or E oU which happens to be a 
measure of the difference between the original data and the value of s calculated by using the deduced 
a/s; it also happens to be Gauss’s criterion for the appropriateness of the chosen functional form 
of F (a smaller value implies greater suitability). [Notes N4.10.1 and N4.10.2] 
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It is important to understand that the method of computation of these formulas is chosen for its mathematical 
convenience and is based on the following assumptions: 

1. Values of jr are without error. 

2. The residuals, v (Eq. 4.5-1) or V (Eq. 4.5-5), are so small that they may be treated like differentials 
( dy or d(ln y)), and the methods of the differential calculus may be used. 

3. The distribution of the errors (6v or <5(ln y)) is Gaussian. 

The mathematical exactness of the formulas, therefore, does not necessarily imply a comparable exactness 
in the description of a practical engineering situation; ordinarily, the formulas merely provide an adequate 
and useful approximate description. 

4.11 Examples of least-squares solutions for polynomials. The succession of these examples will represent 
situations of increasing complexity. Each solution is derived by following the general procedure indicated 
in Sec. 4.10. If 

(a) pairs of measurements (jt,, $,-) are taken; 

(b) the relationship s = F(x) is assumed; and 

(c) measurements of x are assumed to have negligible error; 

solutions will be given, usually for all three cases A, B, and C, as described in Sec. 4.10. 

Unless otherwise noted, the following symbolism will be used (all summations are for / = 1,2 n ): 

For Case A (the a priori uncertainty 5s in j is constant) 

(4.11-1) b k = Djrf.' c k = Lxjs,; d k = Ex-sj; S A = E[£(at,) - s,] 2 

For Case B (the a priori fractional uncertainty 5s/s in s is constant), 

(4.11-2) B k = E(x?/ 5?); C* = E(jc*/i,); D k = Lx*; S B = E[[£ (jc,)/s,] - lj 2 

For Case C (each measurement s, has weight w, ) , 

(4.11-3) b k = Tix k i w j ; c k =Tix k s l w,; d k =Yix k s‘w,; S c = E[F(j:,) - 

Example 4.11 a. If 


F(.r) = a,x 

then, in Case A, with symbolism in (4.11-1), 

a i = C\/b 2 ; e 2 „(a\) = e 2 ai /b 2 ; nelo(s) = e 2 , 


where 


(n - 1)«? 2 ! =d 0 - a,c | 


[Note N4.ll] 

In Case B, with symbolism in (4.11-2), 

a, = C,/B 2 ; el(a,) = E 2 ,/B 2 ; nE 2 M = £ 2 , 
where ( n - 1)£ 2 , = D 0 - a,Ci = n - a,C| 


= S B 

[Note N4.ll] 


69 



In Case C, with symbolism in (4.11-3), 


a, = c,/b 2 ; e](a ,) = e 2 , /b 2 ; b 0 *elo(s) = eh 


where 


(n - \)e 2 ai = d 0 - a,c, 


= 5 C 

(Note N4.ll] 


Example 4.11 b. If 


F(x) = a 0 + a x x 

then, in Case A, with symbolism in (4.11-1), 

flo = (hco ~ b\C\)/D = (c 0 - a x b x )/n; e*(ao) = e\ x »b 2 lD 
a, = (nc | - b x c 0 )/D = (c 0 - a 0 n)/b ] ; el(a x ) = e 2 al »n/D 
D = nb 2 - b\; ne 2 M (s) = e 2 , 

where 


(n - 2)e 2 „ x =do~ (oqCo + a x c x ) 


= S A 


[Note N4.11J 

In Case B, with symbolism in (4.11-2), 

ao = (B 2 C 0 - B X C X )/D = (C 0 - a x B x )/n; ej(oo) = E 2 aX »B 2 /D 
a i = (nC[ — B X C 0 )/D = (C 0 -aon)/Bi; el(a x ) = E 2 aX *n/D 
D = Bo® 2 — ^i»' = £»i 

where 

(n - 2)£oi = n - (ooQ) + <J]C,) 

= 

[Note N4.ll] 

In Case C, with symbolism in (4.11-3), 

a 0 = ( b 2 c 0 - b x c x )/D = (c 0 - a x b x )/b 0 ; el(ao) = e\ x »b 2 /D 
a x = (Vi ~ b x c 0 )/D = (c 0 - a^b 0 )/b x ; el(a x ) = e 2 x *b 0 /D 


D = b(p 2 — b\; 


elo(s) = e 2 aX /b 0 
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where 


(n — 2)e~i — d 0 — (a 0 c 0 + &\C]) 


- $c 

[Note N4.ll] 


Example 4.11c. If 


F(jc) = a x x 4- a 2 x 2 

then, in Case A, with symbolism in (4.11-1), 

= (V'i - b 3 c 2 )/D = (c, - a 2 b 3 )/b 2 ; = e 2 a] »b A /D 

a 2 = (b 2 c 2 - b 3 C\)/D = (c t - a x b 2 )/b 3 ; e 2 „(a 2 ) = e 2 a] »b 2 /D 
D = b 2 b 4 - b\; n»e 2 M = e 2 , 


where 


( n ^) e o\ — do ~ ( a \ c \ + Q2C2) 


= Sa 

[Note N4.ll] 


In Case B, with symbolism in (4.11-2), 

a, = (fi 4 C, - B 3 C 2 )/D = (C, - a 2 B 3 )/B 2 ; e 2 „(a\) = E 2 al »B 4 /D 
a 2 = (B 2 C 2 - B 3 C x )/D = (C, - a { B 2 )/B 3 ; e 2 (a 2 ) = E 2 ol *B 2 /D 
D = B 2 B a - B\; n»E 2 M (s) = Eh 

where 

(n - 2 )£„] = « - (a|C, + a 2 C 2 ) 

= 5 b 

[Note N4.ll] 


In Case C, with symbolism in (4.11-3), 


= (b 4 Ci - b 3 c 2 )/D = (c, - a 2 b 3 )/b 2 ; e 2 0 (ai) = e 2 al »b 4 /D 
a 2 = (b 2 c 2 - b 3 c | ) /£> = (ci - a\b 2 )/b 3 ; e 2 B (a 2 ) = e 2 al »b 2 /D 
D = Z> 2 fc 4 - ftf; b Q »e 2 M (s) = 
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where 


(n - 2) eh =do~ (a x c x + a 2 c 2 ) 


- s c 

[Note N4.ll] 


Example 4 Aid, If 


F(x) = a 0 4 * a x x + a^x 2 


then, in Case A, oq, a x , and a 2 are obtained by solution of the three simultaneous equations 

aobo + ajbj + a 2 b 2 = Cq 
( 4 . 1 1 - 4 ) aob x + a x b 2 + ^3 — c i 

flo ^2 4* 4" a 2 b 4 = C 2 


Also, 


n<?2o(s) = e*, 


where 


(n - 3)eJ, = ^ - (ooc 0 + a,e, + a 2 C 2 > 

[Note N4.ll] 

In Case B, oq> a x , and a 2 are obtained by solution of three simultaneous equations like those of 
Eq. (4. 1 1-4) except that the lowercase b k s and c k s are replaced by uppercase B k 's and C**s, because 
the symbolism of (4.11-1) is replaced by the symbolism of (4.11-2). Also, 

nElois) = Eji 


where 


(n - 3)£^i = n - (oqCq + a,C, + a 2 C 2 ) 

= S B 

[Note N4.ll] 

In Case C, Eq. (4.11-4) holds, except that the b k s and c k 's now have the meanings given in 
(4.11-3). Also, 


V^o(s) = ej, 



where 
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(n 3)ej] — do — (flo^o + a\C\ + a^Ci) 


= 5 C 

[Note N4.1 1] 

Example 4.11e. If n points (x^s,), ( i = 1,2 n) are given, a cubic representation 

F(x) = a 0 + a x x + a-pc 2 + a i x i 

is assumed, and it is required that the ordinate and slope at x t be K, and Y[, respectively, then, 
under the a priori assumption that all points have equal weight, the value of the coefficients which 
satisfy the least-squares conditions, and the associated rms deviations, are given by 

a 3 = (b l c ] - b 2 c 0 )/(b x b 2 - b\) 

a 2= (c 0 - a 2 b 2 )lb x 

a t - Yi - — 3aj*| 

oq= - Y\x\ + ape] + 2apc\ 

( n - 2)eh = d 0 - a 2 c 0 - a 3 c, 

*<*>(•*) = e a] /\In 

where the symbolism for this example is 

*i = Ea, 4 / b 2 = Lafc; b } = L/jJ c 0 = c, = Lftjj,; do = Ltf 

and 

a, = jc, - x,; 0, * (x, - x, ) (x, + 2x,): v, * s, - Y, - Y,a, 

This example is applicable to the piecewise fitting of contiguous segments in order to simulate 
a graphical construction by means of a French curve. In such an application, if the abscissae are 
more or less equally spaced, n is usually 4 and the x x of any segment is taken as the of the 
preceding segment. Y \ and Y x of the segment are taken as the ordinate and slope of the least-squares 
cubic at x„_i of the preceding segment. 

4. 12 Correlation between two variables when each is subject to error. If 

(4.12-1) y = oo + tfi* 

and only the measurements y, are likely to have significant error, the criterion e ol (y) is a reliable measure 
of how well Eq. (4.12-1) agrees with the measured data. However, if both x, and y ( are likely to have 
significant error, a criterion that considers both uncertainties is the correlation coefficient r. Its formulation 
in terms of the quantities given in (4.11-3) is 
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(4.12-2) 


bp c \ ~ ^iQ) 

[(Mo “ c o)~(^0^2 ~ M)]*" 


An alternative formulation that clarifies the meaning of r is 
(4.12-3) r 2 = a x A\ 

where 

a\ is the slope dy/dx deduced when Eq. (4.12-1) is derived on the assumption that x, is without error 
and each measurement v, has weight w,r 

A i is the slope dx/dy deduced when the equation 
(4.12-4) x = Ao + A { y 

is derived on the assumption that y, is without error and each measurement jc, has weight w h The for- 
mula for A i follows from Example 4.11b after symbols jc, and y, are interchanged. 

In Eq. (4. 12-3), the positive sign of Vr 2 is taken if is positive, and the negative sign if is negative. 
Equation (4. 12-3) thus represents the ratio of the two values of dy/dx which are obtained when one first 
assumes y, alone to have error and then assumes x, alone to have error. [Note N4.12.1] 

A common use of the correlation coefficient is to determine which of several variables is more likely 
to serve as a means of estimating the value of a related physical variable. The larger the absolute value 
of r, the better the correlation. The maximum possible value of |r| is unity, which represents perfect 
correlation. However, comparisons among different values of r must always be made with an appreciation 
of the fact that a certain uncertainty exists in r, just as an uncertainty exists in a measure of dispersion 
like e a i (Sec. 3.22). When there are n equally weighted pairs of data, and \r\ > 0.8, the rms uncertainty 
e a (r) in r is given by 


(4.12-5a) e„(r) =/3*(l - r 2 ) ( 1 - r/3) 

where 

(4.12-5b) 0 = ± 1/V(n - 3). 

The positive and negative values of j3 lead to different magnitudes of e a (r). [Note N4.12.2, which refers 
to Ref. 4-1] 

If the relation between x and y is strongly nonlinear and requires three or more constants for its description, 
the correlation coefficient must be replaced by an index of correlation p. This index is treated in Note N4. 12.3. 


4.13 Weight of transformed variables . If 

(4.13-1) y =/(*,M,...) 

and the most probable values of a,b are to be determined from n pairs of measurements (jc^y,), 
(i = 1,2,... ,n), a transformation of the dependent variable may be necessary to facilitate the determination, 
so that 

(4.13-2) s = s(y) =F(X^,R,...) 
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where X,A,B,... may be the same as x,a,b,... or may be simple functions ofx,a,b,... that involve no additional 
unknown parameters. When such a transformation is made, an appropriate change must be made in the 
weight assigned to each observation. 

The weight of an observation is always inversely proportional to the square of the estimated uncertainty 
in the dependent variable. In the method of least squares, one seeks to minimize the sum of the appropriately 
weighted squares of the difference between the measurement and the value predicted by an assumed equation. 
Thus, for Eq. (4.13-1), 

(4.13-3a) ) — vj 2 w, = a minimum 

where 

(4.13-3b) w, = w(y,) = weight of y h 

(All summations are from i = 1 to i = n.) For the transformation (4.13-2), it is necessary that 

(4.13-4a) 'L[F(X i ,A,B ,... ) - j,] 2 W t = a minimum 

where 

(4.13-4b) = W(Sj) = weight of s, 

= (weight of y,)/(5/) 2 
and 

(4.13-5) s’ ® ( ds/dy ),. 

The value of the minimum is a measure of how well the assumed function fits the data. In the case of 
Eq. (4.13-1), the quantities e al or £ ol given by Eq. (4.5-7) provide a measure of the fit. For future use 
of Eq. (4.13-1), the rms uncertainty in y is given by e^iy) or E M (y), where 

(4. 1 3— 6a) ( n - m) (En^e^y) = E[f(x„a,b ,... ) - y,] 2 w, 

or 

(4.13-6b) (n - m)Elo(y) = L{[f(x h a,b t ... )/>,] - l] 2 

the latter equation applying to the case of equal a priori fractional uncertainty in the value of y h 
If the data y f fit f(x it a,b,...) exactly, then e^y) = 0 so that any choice of will lead to the same value 
of the constants a,b,... . The values of e^iy) and a,b f ... become increasingly dependent on the choice 
of Wj as the scatter of the data increases. 

Example 4.13 a. If s = In y 

so that s' = 1/y 

then 

(1) if, a priori , 8y = a constant, the weight of s h by Eq. (4.13-4b), may be taken as 
w, = l/( l/y,) 2 = yj ; 
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(2) if, a priori , by/y = a constant, then, since 

by = v x (a constant) ; 

the weight of y { is 1/y 2 and the weight of S; is, by Eq. (4.13-4b), 
w, = ( l/>',) 2 /( l/y,) 2 = 1 ; 

(3) if, a priori , y, has weight w |( then 

^ = *v,y? 

Example 4.13b. If 5 = log y 
so that s' = (l/y)/ln 10 

then, since In 10 is merely a constant, the values of W i in Example 4.13a also may be used for 
this example. 

Example 4.13c. If y = e™ 2 

and the most probable value of a is to be determined from a number of measurements (x^y,) on 
the a priori assumption that by is constant, then Example 4.13a leads to 

jC(ojc 2 — In y) 2 y 2 = a minimum 


so that 


a = £(;c 2 y 2 *ln y)/£(.x 4 y 2 ) 

(n - \)e 2 al = L(y«ln y) 2 - a£(jt 2 y 2 *ln y) 

"•elo(y) = el\ 

Other examples appear in Sec. 4.14. 

4.14 Examples of least-squares solutions using transformed variables. In this section, some examples 
will be given of least-squares solutions for functions y = f(x) where, for each pair of observations (x f ,y, ), 
(i = l,2,...,n), the dependent measurement y, is assumed a priori , to have a weight w f . Two special cases 
are readily derived from this more general case: 

A. If all measurements of y, are presumed to have the same uncertainty, then one may set all w, equal 
to unity, noting that, thereby, Ew, = n . (All summations are from i = 1 to i = n .) 

B. If all measurements of y, are presumed to have the same fractional uncertainty, then one sets 
w { = 1 /y 2 noting that, in this case, Ewj 2 = n . 

Group 1. Two undetermined constants. Linear relations . In this group of examples, the relation 
(4.14-1) y =f(x,a,b) 
will be transformed into the linear relation 
(4.14-2) s = A + BX 
and the residuals 
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§4.14 



77 




§4.14 


TABLE 4. 14.2. -TERMS IN LEAST-SQUARES SOLUTIONS FOR SOME 
FUNCTIONS WITH THREE UNDETERMINED CONSTANTS 


Example: 

Formula: 

(a) 

y = aa hx + cx2 

<b) 

v = aatb/xi+ic/* 2 ) 

(c) 

v = ai{ 1 +hJt + cr 2 ) 

a 

e A 

t A 

1 IA 

b 

B! In a 

Bi In a 

B/A 

c 

C/ln a 

C/ln a 

C/A 

bj {j = 0 to 4) 

LwxJy 2 

Zwy^/xJ 

LwxJy 4 

c, Q = 0 to 2) 

In v 

E(vv> )2 ln y)/xJ 

ZwxJy 3 

4> 

Ewy^ln 2 ) 

Evv> ,2 ln 2 y 

Zwy 2 

S 

In y 

In y 

My 

X 

Jr 

I ix 

X 

W 

wy 2 

wy 2 

wy 4 

e\{a)te\ x ( s ) 

a 2 (b 2 b 4 -b])/D 

a 2 {b 2 b 4 -b\)/D 

(b^bt-bhHDA*) 

e 2 a (b)le 2 o] (s) 

(brf>4-b%)HD ln 2 a) 

(bfpt-btyiD ln 2 a) 

T 

| 

nr 

i 

J 

ff 

e}(c)lei i (s) 

(b 0 b 2 -b])/(D\n^a) 

{b(p 2 -b*)l(D ln 2 a) 

~(b 2 d 0 -c 2 l )/(DA*) 


Procedure: Given a set of {x it y r 

1. calculate the b s, Cj'\ s, and d 0 from this table; 

2. calculate A, B, C by solving Eqs. (4.14-10); 

3. calculate a, b, c from this table. 

If desired, 

4. calculate e g] (s) from Eq. (N4. 14-11) in Note N4.14.2; 

5. calculate e a (a ), e a (b ), e g (c) from this table. ( D is the 3rd-order determinant of 
the coefficients (the bj s) of (4.14-10).) 


(4.14-3) Vj, ■ v(s,) = A + BXj - s, 

will be assigned appropriate weights in accordance with Sec. 4.13. 

The constants A and B are given by 

(4.14-4a) A = (b 2 c 0 - b lCl )/D 

(4. 14-4b) B = (Vi " b\C 0 )/D 

where 

(4.14-5) D = b<p 2 -b] 
and 

(4.14-6) b 0 = Lw„- b , = LwXo b 2 = LWjXj; c 0 = c, = Lw,X,s, 
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Explicit formulations of these summations for each example are listed in Table 4.14.1, which also lists 
expressions for a and b and, as a matter of incidental interest, the transformed variables s , W, and X. [Note 
N4. 14.1] For ease of tabulation, the index subscript i has been omitted from jc, y , and w in the summations. 
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Group II. Three undetermined constants . In these examples, the relation 
(4.14-7) y = f(x t a,b t c) 
will be transformed into the relation 
(4.14-8) 5 =A + BX+ CX 2 

and the residuals 

(4. 14-9) v„- ■ v(s,) = A + BX, + CX] - s, 

will be assigned appropriate weights in accordance with Sec. 4.13. 

The constants A,B,C are obtained by solution of the simultaneous equations 

Ab 0 4 Bb\ 4 Cb 2 = c 0 

(4.14-10) Ab\ 4 Bb 2 4 0> 3 = c, 


Ab 2 4- Bb-$ 4 Cb$ = c 2 

where the b's and c’s are listed in Table 4.14.2 for each example. The transformed variables s, W, and 
X are also listed, for incidental interest [Note N4.14.2]. For ease of tabulation, the index subscript i has 
been omitted from x , y, and w in the summations. 

4.15 Least-squares solution for a general polynomial. If one assumes 

(4.15-1) y = Yta*x k (k = 0, 1,2. ..,/w) 

and the most probable values of the coefficients a k are to be determined from n pairs of measurements 
(/ = where n > m and each measurement has weight w ( [Note N4.15.1], then the 

coefficients a k may be determined by the following procedures: 

With all summations in Eq. (4.15-2) representing summations from i = 1 to i* = n, we define 


(4. 15-2a) 

bj = Evv^ 

0 = 0,1... 

■ ,2m) 

(4.15-2b) 

Cj = Etv^, 

0 = 0,1... 

.,m) 

(4.15-2c) 

do = TtWjyj 




and construct the set of (m 4 1) equations (the “set of normal equations”) 


aob 0 4 a\b\ 

4 

■ * + afik + • • 

• 4 a m b m Cq 


4 .. 

• + aA+i + -■ 

. 4 a m b m +] Cj 

a of>k + a \bk+\ 

4 .. 

■ + 2k 4 . . 

• + a nt>m + k = c k 

a <Pm 4- Q\b m + 1 

4 .. 

• + ^kf>m + k + •• 

• + aj>lm = Cm 
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The solution of this set yields the (m+ 1) constants a k ( k = 0,1, ...,m). [Note N4.15.2] 
The uncertainty e^(y) is given by 


(4.15-4) 

elo(>’) = e 2 oi/b 0 ; e al = e al (y) 

where 

m 

(4. 15— 5a) 

(n-m-l)e ot 2 = d 0 - £ 


0 


n 

(4. 1 5— 5b) 

= Yj vv ' v ' 2 


o 


[Note N4.ll] 

Example 4.15. A quadratic parabola 

y = a 0 4- G\X + a?x 2 

is to be fitted to the four points (*„y,) = (0,1), (1,0.368), (2,0.135), (3,0.050) in the interval 
0 < x < 3. Then 

A. If each measurement y x is presumed to have equal uncertainty, so that w, = 1, the solution is 

y = 0.988 - 0. 7 1 9x + 0.136&r 2 db 0.03. 

B. If each measurement y { is presumed to have equal fractional uncertainty, so that w, = l/yj, 
the solution is 


y = (0.895 - 0.585* + 0.1013* 2 )(1 ± 0.08). 

The numbers following the ± sign represent e^(y) in case A and E^iy) in case B. 

These two parabolas happen to be approximations to the curve 

y = exp (-*) 

in the ^-interval 0 < x < 3. They are not useful outside that interval. 

4. 16 Least-squares solution for the general case of linearly related undetermined constants . Assume that 

m 

(4.16-1) i(x) = £ aj k (x) 
k= 1 

where each is a function of * that contains no undetermined constants, that there are n pairs of 
measurements (*/,$,) (i = 1,2,... ,n), and that the ith measurement has weight w,. (If all measurements are 
presumed to have equal uncertainty, set Wj = 1 and note that E w, = n; if all measurements are presumed 
to have equal fractional uncertainty, set w, = l/j 2 and note that Ew^f = n. ) All summations not otherwise 
identified will be from i = 1 to / = n. Let 


(4.16-2) /*«/*(*,) 
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and construct the set of m equations 

a x b u 4 a 2 b n 4 ... + a^ ]k 4 ... + a m b Xm = c* 

<* 1^21 + a 2^22 + = c 2 

(4.16-3) a x b k] 4- a 2 ^2 + 4 ... 4 = c k 

a x b m \ 4 a 2 b m2 4 ... 4 4 ... 4 = c m 

where 

(4. 16-4a) b Jk = TiwJjJ kl 

(4. 16-4b) c ; = Ew^,- 

The solution of the m equations (4.16-3) yields the constants a k . The uncertainty ^(j) to be expected 
in future use of Eq. (4.16-1) is given by 

(4.16-5) elo(s) =e 2 al /Lw ( 

where 

m 

(4.16-6) (n - m) e 2 gl = do - £ a*c* 

= EvW 

[Note N4.ll] 
and 

(4.16-7) do = Ewe 2 

(4.16-8) v, = s(x,) - 5 , 

[Note N4.16] 

A similar analysis applies when f k in Eq. (4.16-2) is a function of one or more independent variables 
that are themselves assumed to be without error. The procedure is illustrated by the examples that follow; 
the indicial nomenclature of Eq. (4.16-4) will be changed to suit the particular example. 

Example 4.16 a. Suppose that the f k are the Chebyshev polynomials 

f\ = To = 1; f 2 = T, = at; f i = T 2 = 2x 2 - 1; / 4 = T 3 = 4jc 3 - 3* 

and that the relation 
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5 = a x + a 2 x 4- a{T 2 + a 4 T 3 

is used to describe the temperature s measured by a W5Re/W26Re thermocouple, in the temperature 
range 1000 < s < 2200 K, where x is the nondimensional representation 

*= (2X-X x -X 2 )/{X 2 -X } ) 

and X = emf, in mV , generated when the hot junction is at s kelvin and the cold junction is at 300 
kelvin. 

X x = value of X when s = 1000 K 
X 2 = value of X when j = 2200 K. 

The calibration data used by the manufacturer in 1974 lead to the five points (x„s,) 

(-1,1000); (-0.5,1260); (0,1540); (0.5,1840); (1,2200). 

If each of these points has equal weight, the values 

a i — 1568; <32 = 596; a$ — 32; a 4 = 4 
will yield future measurements of s with an uncertainty 

( 5 ) = 0.7 K 

if there is no change in thermocouple characteristics. 

Example 4.16£. If there are n sets of observations (x^.s,), where / = 1,2 and one assumes 
that 

(4.16-9) s = 0 $ -f Ojx + a^y 

and that each observation s, has weight w h then the constants oq. <i|. a 2 are obtained by solution 
of the simultaneous equations 

bocPo + b xo a x 4- b 0l a 2 = Cqq 

(4.16-10) ^lo^o 4* ^ 20^1 4 b] x a 2 = Cjo 

* 01^0 + b u a x + b 02 a 2 = Cqj 

where 

(4.16-11) b jk - Evv^yf; c jk = 

and all summations are from / = 1 to i = n. The uncertainty e^s) to be expected in future use 
of Eq. (4.16-9) is given by 

(4.16-12a) elo{s) = e\ x /b^ 

where 
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(4. 16- 12b) ( n - 3) e 2 ai = d 0 - (a 0 Coo + a^c i0 + a 2 c m ) 


(4.16-12c) 

= Ew,v? 


[Note N4.ll] 

(4. 16— 12d) 

d 0 = WfS 2 

(4. 16— 12e) 

V, = J (*„>',) - S, 


If all measurements are presumed to be of equal weight, then w t may be set equal to unity in 
(4.16-11), noting that b Q 0 = n. 

If all measurements are presumed to have the same fractional uncertainty, then (4.16-11) may 
be replaced by 

(4. 16-13) b jk = Tix{ y k J sj; c Jk = £jr{y*/ 5, 

and (4.16-12) may be replaced by 

(4. 16- 14a) E^o(5)=£; i A 00 

(4. 16- 14b) (n - 3) E 2 a] = n - (a^c qq + a,c l0 + a 2 c 0l ) 

(4. 16- 14c) = T,Vj 

(4.16-14d) V, = [s(x„y,)/s,] - 1 

[Note N4.ll] 

Example 4.16c, If there are n sets of observations (x^y^Sj), where 
i = 1,2, and one assumes that 
(4.16-15) s = a 0 + d\X + ay + a$xy 

with each observation having weight w it then the constants aQ,a ]k a 2 ,a 3 are obtained by solution 
of the simultaneous equations 

fyxA) + *10^1 + bo\ a 2 + ^\\ a 3 = c 00 
(4.16-16) ^ 10 flo + ^20^1 + ^11^2 + ^21«3 = C {0 
b 0 l a 0 + ^11^] + ^02 a 2 + ^ 12°3 = C 01 
£>D<2q ^21^1 b\2<*2 "h ^22^3 = ^11 

where 

(4.16-17) c jk = llw i s i x } ^ k i 

and all summations are from i = 1 to i = n. 
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The uncertainty e M (s) to be expected in future use of Eq. (4.16-15) is given by 
(4. 16- 18a) zloU) = e 2 o\/bw 

where 

(4 • 1 6- 1 8b) (n — 4) e 2 a] = do - (oqCoo + a]C 10 4- fl 2 c oi + o 3 c u ) 

(4. 16— 18c) =Ew,v? 

(4. 16— 1 8d) do = 

[Note N4.ll] 

(4.16-18e) v, = - Si 

If all measurements are presumed to have the same weight, w, may be set equal to unity in 
(4.16-17), noting that b qq = n. 

If all measurements are presumed to have the same fractional uncertainty, then (4.16-17) may 
be replaced by 

(4.16-19) b )k = Exfyf/s^ c Jk = Ex^yf/s, 
and (4.16-12) may be replaced by 
(4.16-20a) Elo(s) = E 2 ai /boo 

(4. 16-20b) (n 4) E a \ = n ( oo 4~ Q\C\o + q 2 Cq\ 4- UjCn) 

(4.16-20c) =Lv 2 

[Note N4.ll] 

(4.16-20d) V, = [^(Xj.y,)/^] - 1 

Note: This procedure is often convenient for the determination of the values a, b, c in the expression 
(4.16-21) J=fl(l+fo)(l+g) 

although these values are not rigorously the most probable values, because the expression (4. 16-21) 
contains fewer undetermined constants than the expression (4.16-15). The procedure is adequate 
if a r a 2 / (a 3 ao) is sufficiently close to unity to be acceptable for the purposes of the approximation. 
Then a = Oq, b = a x /ao; c = a 2 /ao. 

Equation (4. 16-21) is often a convenient form of representing the dependence of s on the variables 
x and y, when the dependence is slight, and when s appears to vary linearly with each of the variables 
when the other is held constant. For example, s may represent the density of a liquid and x and 
y may represent liquid temperature and pressure. 

Example 4.16d. If there are n sets of observations (x, ,y„s, ), where i = 1,2 n and one assumes 

that 

(4.16-22) s = ao + Q\X + a-jx 2 + a-yy + a ^y 1 
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with each observation s { having weight w,, then the constants a^ t a\ t ai t a^a^ are obtained by solution 
of the simultaneous equations 

^ 00^0 + * 10^1 4“ b 20 a 2 4 ^ 01 a 3 4 b 02 O 4 = c oo 

^10^0 4 b 2 {p\ + £>30^2 4 b u d 2 + b l2 d 4 = c \0 

(4.16-23) ^20^0 4 ^30^1 + ^40^2 4 ^>21 fl 3 4 b 22 a 4 = c 20 

b 0l a 0 4- b\\Q\ 4- b 2 \d 2 4- b^a^ 4 b 03 a 4 = c oi 

^02 a 0 4 b\ 2 ci] 4 - b 22 a 2 4 - 6o3 a 3 4 ^04^4 = c 02 

where 

(4.16-24) b jk = c jk = L/w i s i x } i y k i 

and all summations are from 1 = 1 to 1 = n . 

The uncertainty e^is) to be expected in future use of Eq. (4.16-22) is given by 

(4. 16-25a) tlo(s) = el x /bw 

where 

(4.16-25b) (« — 5) €a\ = d Q — (^cAx) 4 tfjCio 4- d 2 c 2 o 4 ^Cqi 4- d 4 CQ 2 ) 



[Note N4.ll] 

(4.16-25C) d 0 = Lw,sj 

(4. 16-25d) v, = s( Xj.yj) - s r 

If all measurements are presumed to have equal weight, w i may be set equal to unity, noting that 
^00 = 

If all measurements are presumed to have equal fractional uncertainty, then (4.16-24) may be 
replaced by 

(4.16-26) b Jk = Ytxtytys}; c Jk = Exjyf/s, 
and (4.16-25) may be replaced by 
(4. 16-27a) Elo(s) =E 2 ai /b 00 

where 

(4.16-27b) (n - 5) £*, = n - (a^oo + a i c \o + a 2 c 20 + a 2 c 0i + a 4 c 02 ) 

(4.16-27c) = E Vj 

[Note N4.ll] 
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(4. 16— 27d) V { = [s(x hyi )/ Si ] - 1 

Example 4.16e. If there are n sets of observations (x,,>v, 5 ,) , 
where i = 1 , 2 ,..., n, and one assumes that 
(4. 16-28) 5 — y* ( a 0 4 4- flj* 2 + 4- a 4 xy ) 

with each observation of s having the same weight, then the constants a 0 to a 4 are obtained by 
solution of the five simultaneous equations 

fl (A)2 + a \b\2 + @2^22 + ^3^03 + 0 A b^ = Cqi 

^0^12 “b a\b^2 4 - <32^32 ”b ^ 3^13 4“ £ 4/723 = ^11 

(4.16-29) a^ + atb 32 4 fl 2 ^4 2 4 <*3^23 + fl 4^33 “ C 21 

a <A)3 + ^ 1^13 4 fl 2 ^23 + a 3^0A + ^4^14 = c 02 

^ 0^13 4 aj^23 + £ 2 ^33 + fl 3^14 4* ^4^24 = c \2 

where 

(4.16-30) b ]k = Lx>d 

and all summations are from 0 to n. 

The uncertainty to be expected in future use of Eq. (4.16-28) is given by 

(4.16-31a) e^o(s) = e 2 o[ /n 

(4. 1 6—3 lb) ( n — 5)eJj = do — (^oCoi 4 a\C\\ 4 £ 2^21 4 £ 3^02 4 fl 4 Ci 2 ) 

(4.16-31c) do = Ls* 

Example 4*16/. If there are n observations (x,, y if 5 ,), where i = 1, 2, ... , n, and one assumes that 
(4.16-32) s = x a+bJ y c 

with each observation of s having a priori , the same fractional error, then the constants a, b, c are 
obtained by solution of the three simultaneous equations 

a# ^020 + ^*^120 + c# ^on = c 010 

(4.16-33) a # bi 2 o 4“ b # b 22 o 4 c*b U] = Cjiq 

a*b 0 n + b*b\u 4 c # boo 2 = Cqoi 
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where 

(4. 16-34) bju = 'Lx 1 , (In x,) k ( In >',) ' 

Cjki = ^dn J,)x{(ln -r,)*(ln v, ) ' 
and all summations are from / = 1 to i = n . 

The fractional uncertainty E^is) to be expected in future use of Eq. (4,16-32) is given by 
(4.16-35a) E 2 m (s) = E 2 o] /n 

(4. 16-35b) (n - 3 )E 2 at = d 0 - (a»c 0i0 + b»c n0 + c*c 0 01 ) 

(4.16-35c) d 0 = E(ln s,) 2 


4.17 Least-squares solution for a first-order differential equation. Many instrument applications require 
the determination of the constants Y and r of the differential equation 

(4.17-1) T(dy/dx) + y = Y 

where y represents instrument indication, and x usually represents time or distance. In such applications, 
the values of y and x are usually known accurately, but the relation (4.17-1) may be only approximate, 
so that the most probable values of t and Y are of interest; sometimes, only one of these is of interest. 
The two constants may be determined conveniently if the following conditions are met: 

1. Measurements y, are available at equal increments Ax of the independent variable x, so that 

(4.17-2) Xj = (i - l)Ac (» = 1,2,...). 

The initial measurement, y u will be considered to have been made at x = 0, so that X\ = 0 and 
(4.17-3) y, = y(x,) 

2. Successive increments in y, described for brevity as 

(4.17-4) A,*y i + 1 -y, (/ = 1,2,...) 

are all of the same sign and are sufficiently large and sufficiently accurate so that the sequence of 
A, s is monotonic; i.e., the absolute magnitude of each successive A,- is consistently smaller than or 
equal to the absolute magnitude of the preceding A,. 

3. All measurements are, a priori, deemed to be of equal weight. 

Under these conditions, if the last observation is y n+1 , so that there are n values of A„ 

(4.17-5) Y= (b 2 c 0 -b ]Ci )/D 
where 
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(4.17-6) D = nb 2 -b]; 
and 

(4. 17-7a) t/Ax = — 1/ln [1 — d !p)\ 
where 

(4.17-7b) p = (fc,c 0 - nc x )/D. [Note N4.17] 

Here, 

fc, = La, = y n+i - V, 

b 2 = La} = 2 L(y?-yiy i+l ) +y 2 „+\ -y\ 

(4.17-8) c 0 = Ly, 

c, = L,yAi = L(y i y, + ] - yf) 
and all summations are from i = 1 to i = n. 

In an application where many values of (x,,y,) are available but either t or T is suspected of changing 
systematically, though slightly, from the beginning to the end of the sequence of data taking, it is often 
helpful to apply Eqs. (4.17-5) to (4.17-8) only to the readings (four or more) in the vicinity of the region 
of principal interest. Excessively fine subdivision of this region is often undesirable because it detracts from 
the advantage of having larger and, hence, more accurate values of the increments A ( . In the presence of 
undesired high-frequency fluctuations in y„ the use of larger subdivisions is often an effective means of 
satisfying the requirement for a monotonic sequence of A,’s. 

An estimate of the efficacy with which Eq. (4.17-1) fits the data is given by the quantity e g , 

where 

(4. 17-9a) (n - 2) e 2 a = do - c 0 Y + c lP 
(4. 17— 9b) do = Ly} 

However, e 0 does not represent the rms value of the residual 
(4.17-10) v, = [Y+ (y, - y) e -u- '>*/»] - y ._ 

This rms value is usually smaller than e a% but less convenient to calculate. 

If condition 1, that readings be at equal increments, is not fully met, one may choose an increment At 
that is near the average of the actual increments in the available data and then use interpolation to derive 
values of y, at abscissae (i — 1 ) At. A quadratic interpolation formula through four or five points closest 
to (i - 1 )At usually suffices. Such a formula is provided by Example 4. lid, Case A, or by Sec. 4.19. 

If condition 2, that the sequence of A,’s be monotonic, is not fully met, then smoothing by the method 
of Sec. 4.18 may permit meeting the required condition. 3 If the original A,’s had only two signficiant figures 
and were of low accuracy, the use of smoothed values to an additional figure usually improves the accuracy 
of computation of Y and r. 


3 In electrical circuits that will use an analog-to-digital convener to produce numerical data, prior smoothing by use of 
a low-pass filter may be more convenient. 
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Example 4.17a. An illustration of the effects of compromising between n and A, is given by 
presuming some measured ordinates of the equation 

v + dy/dx = 2 


(where i = 1, Y = 2). 

Case 1. If Ax = 0.1 and y, = 2.202, 2.183, 2.165, 2.150, 2.135, 2.122, 2.111, 2.100, then 
n = 7, and Y= 2.003; r/Ax = 9.75; = 0.007. 

Case 2. If Ax = 0.2 and y, =2.183, 2.150, 2.122, 2.100, then 
n = 3, and Y = 1 .999; r/Ax = 5.00; = 0.004. 

Case 3. If Ax = 0.2 and y, = 2.202, 2.165, 2.135, 2.111, then 
n = 3, and Y= 2.011; r/Ax = 4.64; e a = 0.001. 

Example 4.17Z>. An illustration of the effects of rounding is given by assuming some measured 
ordinates of the equation 


y + dy/dx = 0 


(where t = 1, 7=0). 

Case 1. If Ax = 1 and y, = 0.905, 0.333, 0.122, 0.045, then 
n = 3, and Y = -0.000 3 ; r/Ax = 1.000 2 ; e a = 0.001. 

Case 2. If Ax = 1 and y, = 0.90, 0.33, 0.12, 0.05, then 
n = 3, and Y= 0.007; r/Ax = 0.981; = 0.007. 

Example 4.17c. An illustration of the effects of smoothing is provided by assuming that the values 
of y, in Example 4.17a, Case 1 are y f = 2.203, 2.182, 2.166, 2.149, 2.136, 2.121, 2.112, 2.099. 
Then the A,, which range from 0.009 to 0.021, do not form a monotonic sequence. Smoothing by 
the methods of Sec. 4.18, and use of an additional figure in the smoothed values yield the sequence 
y, =2.2027, 2.1828, 2.1651, 2.1499, 2.1348, 2.1225, 2.1106, 2.0995. Then 

Y= 2.004; r/Ax = 9.56; c ff = 0.006. 

4.18 Smoothing of data . For some mathematical operations on empirical data, it is necessary that the 
deviations of the data from a smooth curve be small enough so that the equivalent of a first or second derivative 
may be obtainable. This result may sometimes be accomplished by fitting an appropriate equation to the 
data. This equation may be quite complex and may involve many empirical constants. In such case, better 
accuracy and greater convenience may sometimes be attained by piecewise fitting of a simple quadratic 
or cubic parabola to small groups of adjacent points, overlapping the groups to maintain continuity of the 
derivative. The analytical operation is strictly analogous to progressive use of French curves to draw a smooth 
curve through a large sequence of plotted points. 4 In fact, the graphical operation is often more effective 
and convenient than the analytical one— given a graph on a scale large enough to reveal the least significant 
digit, one may then read the ordinates of the smooth curve at convenient abscissae. 

When unfamiliar data are first acquired, a preliminary graphical construction may also help to reveal 
potential problems, like anomalous points, and to guide the smoothing operation. 

Smoothing becomes particularly convenient when data are obtained at constant increments Ax of the 

independent variable. Let the sequence of data be represented by y t (i = 1,2 n). Let the smoothed 

value of y, be y f . Then 


4 The operation resembles the “cubic spline” method of piecewise construction of a complex curve, but differs from it in the following 
important respect. The cubic spline method creates a curve that treats every point as perfectly accurate and, hence, passes through 
each point; the French-curve method or its analytical equivalent (Example 4. lie) provides smoothing that allows for the imperfect 
accuracy of individual points. 
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(4.18-1) 

V] = 

(19y, 

4- 3y 2 — 

3y 3 4" y 4 ) /20 



(4.18-2) 

>2 — 

(9V| 4- 1 3 \s 4- 

1 2 v 3 4- 6y 4 - 5y 5 )/ 35 



(4.18-3) 

V* = 

[12(y* 

- i + y* 

+ 1 ) + 1 TV* — 3 (»_2 4- 

>'k + 2 

)]/35 

(4.18-4) 

}'n — 1 

= (9v 

n + 1 3y„ 

-i + 12y„_ 2 4" 6y fl _ 3 - 

5v«- 

_ 4 )/35 

(4.18-5) 

}n 

(19y„ 

+ h'n-l 

- 3y n _ 2 4- v„_ 3 )/20 




3 < k < (n - 2) 


Each of these equations represents the result of fitting a quadratic parabola to the four or five points specified 
in that equation. 

This smoothing technique is applicable to empirical data presented in tabular form at constant increments 
of the independent variable. However, such smoothing may be facilitated by applying it to the tabulation 
of some function of v (such as the reciprocal, power, logarithm, or sine) rather than to y itself. 5 

4.19 Interpolation of data. If n pairs of data (jc |f y,) (/ = 1.2 ,..~,n) are available and a value of v(x) 
is desired, where jc is none of the x h a polynomial of the form 

(4.19-1) y = La*r* (* = 0.1 n-1) 

may be fitted to the data. The procedure is to form the n equations 

aob 0 + a l b ] + ... + afi t + ... + a n _ l b„_ l = c 0 

aob } + afa + ... + a k b i+l + ... + = c, 


(4.19-2) 


where 


a ob n - \ + a x b n + ... + <zA + «-i + ... + a n _yb ln _ 2 = c„_j 


bj = Cj = Ex^y,; 0 = 0, 1,2,. . . ,n - 1 ) 

and all summations are from / = 1 to i = n. (Note that b 0 = n and c 0 = Ly r ) The equations are then solved 
for the a k ’ s. The procedure is used most often with n < 5, because it is most effective when a few points 
in the vicinity of x are used to find the interpolated value y(x). 

An alternative formulation of the above procedure is Lagrange’s formula: 

(4. 19-3a) y(x) = P^Lfr/P,] 

where 

(4. 19— 3b) P = II (x - Xi ) (i = 1,2 n) 

i 

(4.19-3c) P, - (x - x )U (Xj - Xj) [/ = 1.2,... (i - !),(/ + 1) n] 


Smoothing, interpolation, and some other operations treated in this chapter are considered thoroughly and authoritatively in Ref. 4-2. 
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Equation (4.19-3a) is the same polynomial as Eq. (4.19-1), but yields only a numerical result, not an 
algebraic formula. 

Example 4 . 19 . Given (A,,y ( ) = (1,9), (2,18), (3,31) to find y(2.5). Then 
Z?q = 3, b\ — 6, b 2 — 14, b~\ ~ 36, — 98, Cq — 58, c \ — 138, C2 = 360 

and the solution of (4.19-2) leads to 

v = 4 + 3a- + lx 2 

so that v (2.5) = 24. 

Eq. (4. 19-3) leads to 

P = -3/8, Py = 1.5 ( — 1 ) ( — 2) =3 
P 2 = 0. 5(1)( — 1) = -0.5 
P 3 = —0.5(2) ( 1 ) = -1 

so that 


y (2.5) = - (3/8) [(9/3) - (18/0.5) - (31/1)] = 24. 

4.20 Significant figures in polynomial representations . When an equation of the form 
(4. 20-la) y = £a*/*U) (k = 0,1,2,...) 

is fitted to a set of experimental data, over the range x { < x < a 2 , a decision is required on the number 
of digits that are signficant in representing each a k . From the relation 

(4.20-2) ( dy/da k ) =/*(*) 

one may deduce that a change in y of magnitude 5y would be produced by a change in a k of by/f k . If by 
is chosen as ^(y), a reasonable representation will result if a k is stated so that the least significant decimal 
digit represents a quantity on the order of 

(4.20-3a) 0.01 ^(y)/|/*| max 

where f is the largest absolute value of f k that occurs in the range x } < x < a 2 . 

The multiplier 0.01 represents a conservative allowance for the accumulation of errors in Eq. (4.20-1) 
and in subsequent computations with it. Very often, a trial computation will show that it is excessively 
conservative. 

In particular, if/* = x k , so that Eq. (4.20-la) becomes 
(4.20- lb) y = La k x k , (k = 0,1,2,...) 
then the expression (4.20-3a) becomes 
(4.20-3b) 0.01 ^ (>•)/!** I™*- 
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Example 4.20. The representation of the emf U of a type E thermocouple in the temperature range 
— 270 < t < 0 °C, relative to a 0 °C cold junction, is given by an equation like Eq. (4.20-1) with 
k = 1, 2, ..., 13 and with e^(U) = 0. 1 /xV. For a g , (4.20-3b) yields 

0.01 x 0.1/270 8 = 3.5 x 10' 23 

so that the least significant decimal bit in a s should represent 1 x 10 ~ 23 . (The value of < 2 g is 
approximately -9 x 10 - 13 , so that x7 g should have 11 digits.) [See Note N4.21, which refers to 
Refs. 4-3 and 4-4.] 

4.21 Improvement of computational convenience . When the variable x may acquire values very much 
larger than unity, a representation like 

(4.21-1) y = La k x k (k = 0,1,2,... ) 

has two characteristics that may impede or inconvenience practical computation: 

(a) As k becomes larger, the number of significant digits required to represent a k may become very 
large; and 

(b) If x has dimensions, each a k has different dimensions, and only a 0 has the dimensions of y. 

If *1 ^ x < x 2 , these deficiencies may be attenuated by transforming the independent variable to 

(4.21-2) X = (2* - Xj - x 2 )/(x 2 - x t ) 

so that X is a nondimensional quantity and -1 s X < 1. Then Eq. (4.21-1) is replaced by 
(4.21-3) y = T,A k X k (it = 0,1,2,... ). 

Each coefficient has the same dimensions as y and, in accordance with Sec. 4.20, the least significant decimal 
digit of A k need not be smaller than 0.01 times the expected uncertainty in y. [Note N4.21] 

If either Eq. (4.21-1) or Eq. (4.21-3) is deduced from data by the method of least squares, exactly the 
same values of y(Xj) and e M (y) will be obtained, unless the computer used is deficient. The transformation 
(4.21-2) serves to improve computational convenience, not accuracy. Accuracy is established by the data, 
not by how the data are represented. However, the more convenient procedure does reduce the number 
of significant digits required in any computing program that is used. 

4-22 Polynomial approximations of intricate functions. Many engineering applications require that a 
relatively simple idealized formula be modified to allow for deviations from the idealized condition. The 
accurate computation of these modifications may require elaborate operations and the use of a computer; 
nevertheless, the final result may often be only a modest modification of the original idealized formula. 
In such case, the final result, once it has been accurately computed, may be rephrased as a simple low- 
degree polynomial in the variables affecting the deviations from the idealized formula. For example, if these 
variables are u,v,w,... and if the idealized formula is 

(4.22-1) s = s 0 (j:,y,z) 

the corrected formula may take the form 

(4.22-2a) s = s 0 (l+f) 

where 


— 


92 



§4.22 


(4 . 22-2b) / = /( x,y,z , u , v, w ) 

and \f\<< 1 . 

When there are two independent variables, the examples of Sec. 4. 16 are applicable to the development 
of such correction formulas. It is desirable that the independent variables in Eq. (4.22-2b) be actually measured 
or otherwise known variables, or else be deduced from parametric equations containing such variables. 

An important conclusion of Eq. (4.22-2a) is that the accuracy of knowledge of s is of the same order 
of magnitude as the accuracy of knowledge of % Thus, in applying Eq. (3.28-3) or (3.28-4), it is only 
necessary to deal with the elementary expression s 0 rather than with the more complex expression of s. 

If the function / can be expressed as a linear combination of a few undetermined constants, these constants 
can be determined by a least-squares procedure, like one of those outlined in the preceding sections. An 
advantage of such least-squares procedures is that they provide the value of e a] as a measure of the 
agreement between the exact relation and its approximation. 

Example 4.22 a. The compressibility factor Z of normal hydrogen gas over the temperature range 
300 < T < 600 K and pressure range 0 < p < 100 atm has been tabulated (Ref. 4-5). The results 
may be approximated within 0. 1 percent of Z by the formula 

Z — 1 + p m (a 0 + a x T + a^p + a&T) 

where a 0 = 8.17 x 10 “ 4 ; a, = -8.12 x 10~ 7 ; a 2 = 4.30 X 10 -7 ; a 3 = -3.34 x 10 _1 ° 

Example 4.22 b. The compressibility factor Z of air over the temperature range 700 < T < 2000 K 
and pressure range 0.5 < p < 100 atm has been tabulated (Ref. 4-5). The results may be approximated 
to within 0.0002 in Z by the formula 

Z = 1 + />*(a 0 + a\T + a{T 2 + ayp + a 4 pT) 
where a 0 = 5.650 x 10 “ 4 , a { = -2.948 x 10 ~ 7 , 

a 2 = 5.52 X lO" 11 , a 3 = 2.35 x 10“ 7 , a 4 = -1.682 x 10“ 10 . 

Example 4.22c. The viscosity rj of N 2 , as tabulated in Ref. 4-5, over the ranges 
300 < T < 1500 K, 1 < p < 100 atm, is given with an rms error of 3 percent by 

tj/tjo = x a + bx *y c 

where rj 0 is the viscosity at T 0 = 273.16 K, x = T/T 0 , y = p in atm, and a = 0.715, b = —0.0142, 
c = 0.0109. 


Example 4.22d. The mass flow rate m of a known gas through a nozzle of diameter d and area 
A in a horizontal pipe of diameter D may be deduced from measurements of the upstream pressure 
p> upstream temperature T, and pressure drop Ap across the nozzle. When Ap < < p, elementary 
application of Bernoulli’s principle yields m = where 

(4.22-3a) m 2 0 = (2 p*Ap/T) ( 1 /R) (A 2 ) 

and R is the specific gas constant. The first factor in parentheses on the right side of this equation 
represents the actual pressure and temperature measurements; the second factor represents gas 
properties; the third factor represents geometric features. 
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In practice, especially where A p is an appreciable fraction of /?, corrections are required for the 
compressibility and other properties of the gas and for geometric features of the installation. The 
complete relation is 

(4. 22 -3b) m = m^KY/sIZ. 

The quantity K is termed the flow coefficient and depends on the design of the nozzle, on the location 
of the points between which Ap is measured, and on the Reynolds number 

(4.22-3c) N Re = 4 m 0 / ( ttDt}) 

of the flow, where 17 is gas viscosity. The quantity Y is a correction for the facts that the flow is 
isentropic but not isothermal, and that the gas is compressible; Y depends on the specific-heat ratio 
7, on A p/p, and on the diameter ratio 0 = d/D . The quantity Z is the compressibility factor of the 
gas (sometimes called the super compressibility factor) and corrects for the fact that the gas is “real” 
rather than “ideal,” so that 

(4.22-3d) p = pZRT. 

Values of K and Y have been tabulated for a variety of standardized nozzle designs (Ref. 4-6). 
For any one nozzle design and any one gas, Eq. (4.22-3b) may usually be written as 

(4.22-3e) m — WoAT 0 ( 1 — F) 

where K 0 is a constant and |F| < < 1 . The percentage accuracy of knowledge of m is of the same 
order of magnitude as the percentage accuracy of knowledge of mo. Thus, a 1 -percent uncertainty 
in measuring p, Ap, or T will result in about 0.5-percent uncertainty in m; a 1-percent uncertainty 
in each of the three independent measurements will result in about a 1 -percent uncertainty in m. 


NOTES FOR CHAPTER 4 

N4.3.1 Each of the following curves passes through the points (0,1), (1/V2, 1/V2), (1,0) with 
6y< 0.001: 

1 . y = cos (tx 2 I2) 

2. y = ax - b sinh * + € ~ x ; a = 8.41, b = 7.47 

3. jc 2 + y 2 = 1 

4. 4 {a — x) (a - y) = 1; a = (1 -I- V2)/2 

5. y = 1 + a: - 2 jc 2 

6. y = 1 — ax 2 *+■ (a - 1 )jc 4 ; a = 3 - 2 V2 

7. y = 1 + otx + (a — 1 )x 3 ; a = 3 - 2 V2 

Furthermore, over the range 0 s x < 1 , the average disagreement between the ordinates of the first 
two curves is 0.004, and the maximum disagreement is less than 0.014. 

N4.3.2 As an example, over the range 0 < x < 2, the exponential y = e ~ x is approximated by the 
quadratic parabola 


y( x) = 0.9813 - 0.8155jc + 0.200Q* 2 
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so that the difference between the ordinates of the two curves does not exceed 0.02. However, at 
jc = 2.5, the difference is 0.1 1 and at jc = 3, the difference is 0.28. 



N4.8 The log scale of semilog (or log) paper is ordinarily marked with the values of y (or of v and 
of x). However, when the span of log v (or log x) is a fraction of one decade, it may be more convenient 
to plot log v (or log x) on a uniformly spaced scale of cooidinates. 


N4.10.1 Occasionally, the rms uncertainty e a (aj) will also be stated. However, the criterion for 
determining the number of significant figures to be used in representing any a } should be the criterion 
described in Sec. 4.20. 


N4.10.2 A useful property of the least-squares method is that approximations made in the assignment 
of weights, for reasons of mathematical convenience, usually cause only small and acceptable increases 
in the value of e al or Ed- However, in this text, no such approximations will be made unless they 
are explicitly identified. 


N4.ll The two equations for e 2 a] or E 2 ol are alternate expressions of the same quantity. The 
advantage of the first expression is that the first summation on the right side can be computed at 
the time that the original data are entered into any computer program; the disadvantage is that the 
right side represents a very small difference between two relatively large quantities, so that each 
of these quantities must be computed with very high arithmetic accuracy. The advantage of the second 
expression is that it represents the direct summation of very small quantities, so that high arithmetic 
accuracy is not required; the disadvantage is that the original data must be reentered into a computer 
program after the simultaneous equations have been solved. 


N4.12.1 Still another way of defining r is 


(N4. 12-la) 
where 

r 2 = 1 - S 2 (y)/a 2 (y) = 1 - S 2 U)/a 2 (*) 

(N4. 12-lb) 

5 2 (y) = E(a 0 + a, Xj - >,) 2 »v, = d 0 - oqc 0 - a,C| 

(N4.12-lc) 

S 2 {x) = E(/lo + A h y, - Xj) 2 Wj 

(N4. 12— Id) 

a 2 (y) = E(y, - y 0 ) 2 w, = d 0 - cl/b 0 

(N4. 12- le) 

a 2 (x) = L(jr, - x 0 ) 2 w, 

(N4. 12— 1 f) 

y 0 = E(H' i y I )/Lvi' / ; x 0 = 


The b k ,c k ,d k are given by (4.11-3), and all summations are for i = 1,2 

N4.12.2 R.A. Fisher (Ref. 4-1) has pointed out that the distribution of errors in r, especially when 
r is close to unity, is not of Gaussian form and is skewed (i.e., it is not symmetrical about the most 
probable value). However, the transformation 

(N4.12-2) z = (l/2)ln[(l +r)/( 1 - r)] =tanh _1 r 

leads to an approximately Gaussian distribution of z, so that e a (z), e a (z), and e p (z) are related 
by (3.18-2). The nonlinearity of Eq. (N4. 12-2) is so pronounced that a change A z * e a (z) produces 
a change A r in r that is radically different from the change - A r produced by a change — A z. When 
\r\ > 0.8 and the n pairs of data are equally weighted, this nonlinearity is adequately accounted for 
by Eq. (4.12-5). 
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The proportionalities between the uncertainties in z are 

e a (z): e a (z): c p (z) = 1: 0.80: 0.67 

where, as deduced by Fisher, 

e a (z) * 0. 

N4.12.3 If 

(N4.12-3) y=f( x ), x = F(y) 

are nonlinear functions of x or y, the correlation coefficient r is no longer useful. However, two 
other criteria of correlation may be used. They are the indices of correlation p x and p v defined by 

(N4.12-4a) p? = 1 — S 2 (y)/cr 2 (y) 

(N4.12-4b) p 2 = 1 - S 2 {x)/o 2 (x) 

where 

(N4. 12-4c) S 2 (y) = - y,] 2 w, 

(N4.12-4d) S 2 (x) = L[F(y,) - x,] 2 w, 

(N4.12-4e) a 2 (y ) = E(y, - y 0 ) 2 >v, 

(N4.12-4f) <r 2 (x) = L(x, - x 0 ) 2 w , 

(N4.12-4g) >’ 0 = 2 /Sw,; x 0 = E(w,x,) /Ew, 

These equations are parallel in form to Eqs. (N4.12-1). The distinction between the two groups 
of equations is that p x and p y are usually different; they are equal to each other only when f(x) has 
the linear form of Eq. (4.12-1). 

The index of correlation may be used to determine whether a variable x is better correlated with 
y than is some other variable The comparison is possible only when the functional form/remains 
the same. Thus, if 

y = f(x) yields p y = 0.9 
and 

y=/(() yields p y = 0.95 

then the correlation between £ and y is better than the correlation between x and y. 

However, for any set of n points (x,, v,,w,), the value of p v may always be made to be closer to 
unity by increasing the number of undetermined constants in/(jc); if the number of these constants 
is equal to n, then p x and p y are identically equal to unity. 

N4.14.1 The uncertainty of principal interest is e M (y), or E M (y), which represents, respectively, 
the rms uncertainty in y or the rms fractional uncertainty in y, which may be expected in future 
use of Eq. (4. 14-1) in the x-interval (span) of the original data. These uncertainties are most readily 
expressed in terms of the quantity, e a] ( y ) or £ ol (y), which represents the rms value of the res iduals 
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(N4.14-1) 

v v , = v(>',) =f(Xj,a,b) - y, 

or 


(N4.14-2) 

v y , = V(y,) = [f(x„a,b)/y,] - 1, 

the latter residual being used when, a priori , all y, are deemed to have constant fractional 
uncertainty. Then 

(N4.14-3a) 

elo(y) = elx/hwi 

where 


(N4.14-3b) 

(n - 2)e\\ = Ev*; e aI = e ol (y); 

or 


(N4.14-4a) 

c 

'''X 

kf 

II 

k) 

where 


(N4. 14-4b) 

(n - 2)E 2 cl = Lv%; E a] = E al {y). 


A convenient substitute for Eq. (N4.14-3b) or Eq. (N4.14-4b), in certain cases, is obtained by 
using the intermediate quantity e al (s), given by 

(N4.14-5) (n - 2)el, (s) = d 0 - Ac 0 - Be, 

where 

(N4.14-6) d 0 = L(W,sj). 

Examples of such cases are the following: 

1 . When the transformation is such that s = y, then 

e,\ * e ai (y) = e al (s). 

2. When the transformation is such that s = In y, and all v, are deemed to have the same 
fractional error, then 


E 0 \ = E ol (y) = e aX (s). 

3. When the transformation is such that s = 1 /y, and all y, are deemed to have the same 
fractional error, then 


E a] = E a , (y) = E a] (s) = e aX (s). 

The relative merits of using Eq. (N4. 14-5) in place of Eq. (N4. 14-3b) or (N4. 14-4b) are discussed 
in Note N4.ll. 


N4.14.2 The uncertainties of principal interest, e^(y) and E^y), which represent the uncertainties 
to be expected in future use of Eq. (4. 14-13) in the x-interval (span) of the original measurements, 
are describable in terms of the residuals 
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(N4.14-7) v„ = v ( v, ) =f(x„a,b,c ) - y, 
or 

(N4.14-8) 

Then 

(N4. 14-9a) 
where 

(N4. 14-9b) (n - 3)e 2 o] = £(vj,w,) ; 
or 

(N4. 14- 10a) Elo(y) = E 2 Jn ; E ol = £ ff] Cv) 

where 

(N4.14-10b) ( n - 3 )£ 2 , = £(*$*,) . 

A convenient substitute for Eq. (N4. 14-9b) or Eq. (N4.14-10b), in certain cases, is obtained by 
use of the intermediate quantity e„i(j), given by 

(N4.14-1 1) (n - 3)e 2 ,(j) = d 0 - Ac 0 - Bcf - Cc 2 
where 

(N4.14-12) do = £(»>?). 

Three such cases are those described in Note N4.14.1: 

1. If i = y, then e„, m e al (y) = e oI (j). 

2. If s = In y, and dy/y is presumed constant, then £ oI = E al (y) = e al (s) 

3. If s = 1/y, and 6y/y is presumed constant, then £„, * E al (y ) = £ oI (s) = <.„,(*). 

The relative merits of using Eq. (N4. 14-11) in place of Eq. (N4.14-9b) or (N4.14-10b) are 
discussed in Note N4. 11. 

N4.15.1 If each measurement is presumed to have the same uncertainty, then w, should be set equal 
to unity, so that Eiv, = n. 

If each measurement is presumed to have the same fractional uncertainty, then w, should be 
replaced by 1/y, 2 , so that Lwyif = n. 


V y , - v (}',) = [f(x,,a,b,c)/y,] - 1 . 
eioiy) = c 2 |/Evty e a] S e al (y) 
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N4.15.2 Each of the coefficients a k has an rms uncertainty e a (a k ). A general procedure for finding 
these is indicated in Note N4.16. Only a change in the formal symbolism is required. 

N4.16 Each of the coefficients a k has an rms uncertainty e ff (a k ). The procedure for finding these 
uncertainties is as follows. Let D represent the determinant of the coefficients on the left side of 
the set of equations (4.16-3) 

. . . b Un 
D = 

1 • * * 

and let D kk represent the cofactor of the diagonal term b kk . Then 

e](a k ) = (D kk /D)e 2 al . 

This equation is exact if the weights do not depend on the y's, as in Case A, Sec. 4.11; it is a close 
approximation if the weights do depend on the y’s, as in Case B, Sec. 4.11. 

For example, in Example 4. lid. Case C, 

?o(a o) = e 2 a \*(b 2 b 4 -b 5 )/D 

e;(a { ) = e 2 a ^(bob 4 -bj)/D 

e;(a 2 ) = e 2 ol *(bob 2 -bi)/D 

where 

b 0 b\ ^2 

D = b ] b 2 b 3 
b 2 b } b 4 

N4.17 Either Y or t may be computed without knowledge of the other. 

If Y is of principal interest, but t is also of interest, then Y may be computed first by Eq. (4. 17-5), and 
then Eq. (4.17-7b) may be replaced by 

(N4.17-1) p = (Yb } - Cl )/b 2 

If r is of principal interest, but Y is also of interest, then r may be computed first by Eq. (4.17-7), and 
then Eq. (4.17-5) may be replaced by 

(N4. 17-2) Y = (c 0 4- pb t )/n 

Use of Eqs. (N4.17-1) and (N4.17-2) involves fewer computational steps than use of the equations they 
replace. This fact may be significant when a real-time (on-line) computer program is to be used. 
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N4.21. A further moderate reduction in the number of significant figures is possible by replacing Eq. (4.21-3) 
with 

(N4.21-1) y = La k T k 

where T k (X) is the Chebyshev polynomial defined by 

T k + 2 = 2XT k+] — T k (k = 0,1,2 m) 

(A4.21-2) T t =X 
7 - 0=1 

and X is given by Eq, (4.21-2). The Chebyshev polynomial has the property that 
1 < T k < 1 if - 1 < X < 1 

The advantage of Eq. (N4.21-1) over Eq. (4.21-3) becomes more significant as m becomes larger. 
As an example, the emf y, in microvolts, generated by a type E thermocouple, when the reference junction 
is at 0 °C and the other junction is at x °C, in the range - 270 < x < 0 'C, is given to within 0.01 
microvolt by Eqs. (4.21-1), (4.21-3), and (N4.21-1) when the coefficients have the following values: 


k 

Ok 

4t 


0 

0 

-6713.695 

-5830.645 

1 

58.695 857 799 

5278.258 

5015.620 

2 

0.051 667 517 705 

1685.958 

897.483 

3 

-4.465 268 3347 x 10 _4 

-296.232 

-94.451 

4 

-1.734 627 0905 x 10 ” 5 

131.536 

15.190 

5 

-4.871 936 8427 x 10 -7 

-71.680 

-3.898 

6 

-8.889 655 0447 x 10 

-279.488 

0.563 

7 

-1.093 076 7375 x 10" 10 

154.880 

0.360 

8 

-9.178 453 5039 x 10“ 13 

710.272 

-0.265 

9 

-5.257 515 8521 x 10 - ' 5 

-505.088 

0.064 

10 

-2.016 960 1996 x 10“ 17 

-689.664 

0.045 

11 

-4.950 213 8782 x 10 _2 ° 

574.464 

-0.128 

12 

-7.017 798 0633 x 10" 23 

237.568 

0.116 

13 

-4.367 180 8488 x 10 -26 

-217.088 

-0.053 


The original data and the coefficients a k are from Ref. 4-3. The coefficients a k are from Ref. 4-4, which 
pointed out the usefulness of the Chebyshev polynomials. 
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CHAPTER 5. SOME BASIC ELEMENTS OF 
STATIC PERFORMANCE 


5.0 Terms characterizing instrument performance . Some terms that may be applicable to characterization 
of an instrument’s performance are listed here. 

A. Characteristics expressed in units of the physical variable being measured. 

(1) Range: a statement of the limits y min and y max . (Sec. 3.4) 

(2) Span: the value of \y m - y min |. (Sec. 3.4) 

(3) Readability: the smallest change in indication that can be detected. 

(4) Repeatability: the extent to which the same indication will be obtained when the value of the 
physical variable is altered and then restored to its original value. The nature of the alteration must 
be stated. Repeatability may be different in the following different circumstances: 

(a) the reading is made immediately after restoration. 

(b) the reading is made several minutes after restoration. 

(c) the reading is made several days after restoration. 

(5) Sensibility: the smallest change that can be detected reliably. This value is usually larger than 
the readability. It is often synonymous with repeatability. It is a convenient descriptive term, even 
though its definition is not precise. 

(6) Inaccuracy: the difference between the indication and the true value of the quantity it is intended 
to measure, as treated in Chapter 3. It usually represents the random error of measurement and is 
stated as the probable, average, or rms error (standard deviation), or as a percentile (Sec. 3.20), or 
as a limit of error (Sec. 3.7). 

B. Other characteristics. 

(7) Sensitivity: the value dy/dx, where x represents the physical variable being measured, and y 
represents the indication. This indication may be in any one of various units: scale divisions, counter 
readings, voltage, current, pressure, or the same as the units of x . 1 

(8) Nonlinearity: in instruments whose indication y is intended to be proportional to some function 
u of the physical variable x being measured, the extent to which that proportionality is actually achieved. 
It is usually described either as 

(a) the variation in dy/du over the range of the instrument, or 

(b) the maximum difference between the indication and the ordinate of a straight line approximating 
y(w), both values taken at the same abscissa. The various ways in which the straight line 
may be drawn have been treated in Sec. 4.9. The maximum error may occur at a limit of the 
range or at some intermediate point. The nonlinearity is usually expressed as a percent of 
span, or, if the lower limit of the range is zero, as a percent of full scale. 


•Industrial-process instruments in the U.S.A. are often designed to provide standardized indications like 1-5 V, 4-20 mA, or 3-15 
psig, in order to facilitate operation of a variety of control elements of different manufacturers. The lower limit of these indications 
corresponds to the value x = 0. 
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5.1 Pointer and scale instruments. Such instruments are useful when 

(a) the variable being observed fluctuates slightly and randomly with time, and knowledge of the 
existence, periodicity, and order of magnitude of the fluctuation is valuable; 

(b) a particular value of the indication (the “set point“) is desired and it is of value to know not 
only whether or not the set point has been reached, but also the degree of deviation from the set 
point and, if the deviation is unsteady, how it varies with time. Instead of a set point, there may 
be a “working band“— a band of contiguous values. 

(c) rapid visual perception of the approximate indication is necessary 

(d) there are advantages in cost, reliability, or size over instruments using counters, lights, or flags. 

Example 5.1a. When a null galvanometer is used to indicate the balance point of a Wheatstone 
bridge, the approach to the balance point may be made smoothly and any drift in indication due 
to self-heating may be detected. 

Example 5.1ft. When a number of 300-degree or 360-degree circular scale instruments is used 
on an instrument panel, an appropriate group of instruments may be arranged so that normal (bogey 
or nominal) indication is represented by the 3-o’clock position of each pointer. Any deviation from 
the bogey conditions is then quickly apparent. 

Example 5.1c. In the flight test of an airplane during an acrobatic maneuver, the pilot may be 
asked to attain and hold a desired acceleration for a brief period of time. An indicating accelerometer 
can provide the required indication of the approach, maintenance, and recovery from this condition. 
The accelerometer becomes part of a feedback control system using a human operator. [Note N5. 1] 

5.2 Scales. When accuracy is important and quick readability is not, the following sources of error may 
be present in mass-produced scales. Their relative prominence affects the nature and extent of the instrument 
calibration that is required. 

(1) Play between pivot and socket, in instruments using this more rugged construction 

(2) For elastically supported pointers, the relevant spring errors listed in Sec. 5.6 

(3) The fact that the pivot is not at the center of a circular scale, dial, or drum, so that a systematic 
error results 

(4) Improper location of major scale intervals 2 

(5) Improper subdivision of major intervals, involving: 

(a) systematic errors caused by an incorrect interpolation process 

(b) random errors caused by the marking machinery 

(6) Shrinkage and distortion caused by a duplicating process 

(7) Parallax in reading: 

(a) For scales without mirrors, the error due to parallax that may reasonably be expected is a 
distance on the scale equal to one-fifth of the distance between pointer and scale. 

(b) For scales with mirrors and knife-edge pointers, the parallax error is negligible in comparison 
with the error listed under “inadequate readability.” 

(8) Inadequate readability. A working rule is that, neglecting parallax, the readability is not better 
than the largest of the following three quantities: 

(i) a distance of 0. 1 mm on the scale 

(ii) one-fifth of the thickness of a knife-edge pointer 

(iii) one-tenth of the smallest scale interval between adjacent markings 

5.3 Force measurements . Physical quantities in mechanics and electricity whose dimensional representation 
in the force-length-time system explicitly includes the dimension of force are often measured, directly or 


2 Usually, a precision instrument is marked to be correct at major scale intervals, during the manufacturer’s calibration or adjustment. 
Subdivision between these intervals is then performed by a machine in accordance with a predetermined interpolation rule. 
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indirectly, through a force measurement. This measurement, in turn, is commonly made by one of the 
following methods: 

A. Comparison with the force exerted by gravity on a known mass (lever-type balance, or pendulum) 

B. Measurement of the current through a coil or solenoid in the field of a permanent magnet 
(electrobalance) 

C. Measurement of the deflection of a spring. 

5.4 Errors in dynamometers using gravity. Measurements using method 5. 3 A are subject to errors caused 
by 

(a) uncertainty in the knowledge of the value of the local acceleration of gravity [Note N5.4] 

(b) incorrect length of lever arms, because of imperfections in manufacture 

(c) friction in bearings or knife edges 

(d) play in bearings or knife-edge seats 

(e) bending of levers under load 

(0 systematic change in lever-arm geometry with load 

(g) change in lever-arm lengths caused by temperature or by temperature gradients 

(h) uncertainty or systematic change in pivot-point location when elastic knife edges (Emory knife 
edges) are used 

(i) resisting torque or force when elastic knife edges are used 

(j) uncertainty in location of the effective line of action when flexible tapes are used in contact with 
pulleys or cams, because of flexural resistance of the tape or of presence of dirt between the tape 
and the surface of the cam 

(k) uncertainty of effect of air buoyancy on the components 
([) electrostatic forces on components 

5.5 Errors in electrobalances. Electrobalances (method 5.3B) intrinsically depend on the relation between 
force and coil current. This current is usually determined by measuring the voltage drop across a stable 
precision resistor in series with the coil. The measurements are subject to errors caused by 

(a) the systematic effect of temperature on the remanence of the permanent magnet 3 

(b) change in coupling between the coil or solenoid and the magnet, due to self-heating or change 
in the direction of gravitational acceleration (if the electrobalance is not of the rotary type) 

(c) reduction of magnet strength (demagnetization) caused by mechanical impact, excessive vibration, 
or an external magnetic field 4 

(d) any of the sources of error listed in Sec. 5.4 for method 5.3A, that may be applicable 

5.6 Errors in springs . Instruments using method 5.3C are usually designed to have a substantially linear 
relation between the scale indication and the quantity being measured. Terms specifying the performance 
of instruments in this group are 

(a) Nonlinearity. See item (8) in Sec. 5.0. 

(b) Hysteresis . In its most general sense, the greatest difference between the ordinates, at the same 
input, of the two curves of indication versus input, one curve generated by steadily increasing 
the input from its lower limit to its upper limit, the other curve generated in the reverse direction 
and also over the full span. This general definition includes three effects: 

(i) backlash 

(ii) dry friction 5 

(iii) true elastic hysteresis of the spring material 


3 Properly prepared Alnico may have a temperature coefficient of remanence smaller than 0.0003/ °C. 

4 Properly prepared Alnico is highly resistant to such influences. 

Although the coefficient of friction may be low when two surfaces are in relative motion, the coefficient may be many times larger 
at the instant when the motion is initiated from a position of rest. This phenomenon is evident even in lubricated ball, roller, and 
journal bearings. 
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To most users, only the combined result of these three effects is of interest. [Notes N5.6.1 and 
N5.6.2] 

Hysteresis, like nonlinearity, is usually described as a percent of span. 

(c) Zero drift . The difference between the indication immediately after removal of load (generally 
full-scale load) and the “final” indication some time later. It is usually expressed as a percent 
of span. [Notes N5.6.2 and N5.6.3] 

(d) Full-scale drift (creep). The difference between the indication immediately after application of 
full-scale input and the “final” indication some time later. It is usually expressed as a percent 
of span. [Notes N5.6.2 and N5.6.3] 

(e) A common, convenient specification is that combined nonlinearity, hysteresis, and drift of an 
instrument shall not exceed a stated percent of the span (or of full scale). If the span is much 
larger than full scale, it is necessary to specify explicitly whether hysteresis should be based upon 
a change equal to full scale or upon a change equal to the span. 

Since nonlinearity is a systematic error, it may be corrected for with slight loss of convenience 
but with gain in accuracy. However, hysteresis and drift must be considered as random errors, 
since their sign and magnitude depend on the prior history of change in the input. 

(f) Temperature coefficient of sensitivity. With sensitivity defined as in Sec. 5.0.B.7, this property 
may be described in either of the following ways: 

(i) the change in sensitivity, per unit change in temperature, at the nominal (bogey) operating 
temperature, or 

(ii) the change in sensitivity over a stated range of operating temperatures; this is also 
expressible as a mean coefficient over the stated range. 

For a linear spring, the temperature coefficient of sensitivity is usually the temperature 
coefficient of the modulus of elasticity of the spring material plus the temperature coefficient of 
linear expansion of the material. [Note N5.6.4] 

(g) Temperature effect on zero. The principal component of this effect is usually a systematic variation 
with temperature, but there may also be a random component that depends on the prior temperature 
history of the spring. The systematic effect may be described in one of the following ways: 

(i) the derivative of zero shift with respect to temperature at the nominal (bogey) operating 
temperature 

(ii) the average change in zero, per unit temperature increment, over a stated temperature 
range. 

An overall specification that includes both systematic and random effects is the maximum shift of 
zero when the temperature is varied over a stated range in any arbitrary manner. 

(h) Pressure effects. Ambient pressure may affect both sensitivity and zero. Systematic effects can 
be described as a percentage change in sensitivity and an absolute change in zero, per unit change 
in pressure; such effects may be corrected for. (Differential-pressure gauges often show such 
effects.) Random effects are best described as the maximum change in sensitivity or zero over 
a stated range of ambient pressure. 6 

(i) Some items listed in Sec. 5.4 apply also to dynamometers using springs. 7 

5.7 External physical variables affecting steady-state indication . The environment in which the instrument 
is used may affect its indication or operation. Among such environmental factors are 

(1) Temperature and temperature gradients 

(2) Ambient or barometric pressure 

(3) Steady acceleration (e.g., position or attitude relative to the Earth) 

(4) Vibration or acoustic noise 


6 Mechanical oscillators, like tuning forks, if not hermetically encapsulated, are subject to effects of ambient air pressure, but these 
effects are due principally to the change in mass of entrained gas rather than to the change in spring stiffness or in dimensions. 

7 The terms reproducibility or repeatability, when applied to springs, usually include random effects, but exclude systematic ones 
which may be corrected for. 
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(5) Heat exchange with surroundings (radiative, conductive, or convective) 

(6) Correlated electric or magnetic fields (hum) or currents, such as those due to ac power lines 

(7) Uncorrelated electric or magnetic fields (noise), such as those created by adjacent instruments; 
radiofrequency interference (RFI) is one appellation for such fields in the radio spectrum 

(8) Optical, X-, 7“, neutron, or cosmic radiation 

(9) Ambient-gas composition, including humidity 

(10) In electrically powered instruments, deviations in the power supply from the nominal (bogey) 
values of voltage, frequency, or wave shape 

(11) In pneumatically powered instruments, deviations from the nominal value of gas pressure, or the 
presence of particulates (dust) or water in the gas supply. 

An instrument user may either 

(a) specify the limits of any one of these factors, between which satisfactory operation will be 
assured, or 

(b) specify limits on the magnitude of any systematic effects of these factors, or 

(c) require knowledge of the quantitative magnitude of an effect, so that a systematic correction may 
be applied by the user, or 

(d) take corrective measures to reduce an effect to an acceptably small magnitude. 8 

Example 5.7a. A specification for an electrically powered instrument may require that the change 
in indication shall not exceed a specified amount when there is any combination of line voltage between 
105 and 130 volts, line frequency between 59 and 61 hertz, and harmonic distortion between 0 and 
5 percent. 


Example 5.7 b. A specification for a differential-pressure gauge with a range of 1 unit of pressure 
may require that the effect of ambient temperature shall not exceed 0.0005 unit/°C in the range 
10 to 30 °C, that the effect of ambient pressure shall not exceed 0.01 unit/bar in the range 0.5 to 
1.5 bar, and that a 30-degree tilt of the gauge in any direction from its normal operating attitude 
shall not change the indication by more than 0.001 unit. 

Example 5.7c. A specification may require a statement of the mean sensitivity coefficients for 
each of the effects listed in Example 5.7b. 

Example 5.7*/. For a pneumatically powered instrument, the user may provide a pressure regulator, 
a filter, and a dryer for the gas supply. 

Example 5.7c. For an instrument intended to measure low-level emf s, the manufacturer and the 
user may take steps 

(i) to reduce the presence and the effects of thermal emfs [Notes N5.7.1 and N5.7.2] 

(ii) to reduce the electrically and magnetically induced effects of alternating-current power 
lines, particularly if these provide power for the instrument [Note N5.7.3] 

5.8. Long-term effects of external physical variables . Sometimes, the effect of an external physical variable 
may become significant only after a long time of exposure. Such long-term effects cannot usually be included 
in an instrument specification, but their presence and magnitude influence instrument selection because they 
affect the frequency with which recalibration and routine maintenance will be required. 

Example 5.8a. The indication of a thermocouple in an installation where a section A L has been 
in a high-temperature gradient A6/AL for a long period of time may change significantly after the 
gradient has been removed or after the section AL has been replaced by a new section because the 
immersion depth has been changed. 


Approach (c) usually yields reductions in cost and in acquisition time over approaches (a) and (b), but it also involves losses in 
convenience and in ease of data analysis and use. 
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Example 5.8Z>. The calibration of a thermocouple which has Rh or Re as a component of one 
of its wires may have changed significantly after months or years of exposure to a high thermal 
neutron flux. 

Example 5.8c. Oxidation of the surface of a radiation shield or of a pyrometer wire may alter 
the radiation correction of a pyrometric or radiometric device. 

Example 5.8d. Accretion of dirt on the window of an optical pyrometer, nephelometer, or 
radiometer may alter the instrument’s indication. 

Example 5.8c. Accretion of dirt on the surface of a nozzle or of a pipe may alter the calibration 
factor of a head-type fluid flowmeter or of a boundary-lay er-type calorimetric fluid flowmeter. 

5.9. Effects of the spatial profile . If there is a spatial variation of a physical quantity u, the spatial integral 
of u (the integral of u over a linear distance, area, or volume) may be of interest in itself or may be used 
to provide the average u of u. Such an integral or average may be obtainable by one or a combination of 
the following methods. 

(1) A single local measurement of u at a location at which u has the same value as u. This method 
is useful when the profile is known. 

(2) Use of an instrument that directly provides an indication that represents the correct integral of u . 

(3) Measurement of the profile by a continuous traverse with an instrument that provides the local 
value of u . This method usually is practical only when the traverse is along just one of the orthogonal 
dimensions (e.g., straight line or arc) of the relevant coordinate system and when the local values 
of u change negligibly during the time interval consumed by the traverse. 

(4) Simultaneous discrete local measurements of u at locations chosen so that the summation of these 
measurements, with appropriate weighting, yields the desired integral. 

Each of these methods will be treated in the sections that follow. The spatial distribution of u may be 
represented 

(a) in rectangular Cartesian coordinates, by u(Jt), w(jc,y), or u(x,y f z); 

(b) in polar or cylindrical coordinates, by u(r,$ ), or u(r,0,z)- 

5.10. Spatial average from a single local measurement When it is known that a profile u(x) could be 
represented by a simple analytical formula involving only one constant or parameter that changes with the 
average velocity u, there may be one location jc 0 at which u( jc 0 ) = u for all values of u that may be 
experienced. It may be necessary to specify certain physical conditions that will ensure this favorable situation. 

If the profile is expected to vary between two limiting forms, it may be possible to find some intermediate 
value of x 0 which will yield equal absolute values of the positive and negative errors that occur when the 
respective limiting profiles exist. 

Example 5.10. To determine the volumetric velocity (bulk velocity) of fluid flowing unidirectionally 
in a circular pipe of radius R, if the fluid fills the pipe, and if the flow is axially symmetric, one 
may assume that the average linear velocity u is equal to the local velocity at some radius r 0 whose 
value depends on the nature of the flow. 

(i) If the flow is fully developed and laminar, u(r 0 ) will be equal to u (bulk velocity /area) 
when 

r 0 /R= 1/V2 [Note N5. 10.1] 

(ii) If the flow is fully developed and turbulent, u(r 0 ) will be equal to u when 

r 0 /R = 0.758 [Note NS. 1.0.2] 
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(iii) Consequently, if the flow is expected to range from fully developed laminar to fully 
developed turbulent, a measurement of u at 

r 0 /R = 0.715 

will yield a fractional error in mean-velocity measurement, such that 
|[1 -u(r 0 )/u ] | < 2.3 % . 

5.11 Spatial integration by physical means. An appropriate geometric configuration may be used to produce 
a measurable effect representing the desired integral. 

Example 5.11a. The heat of combustion of a chemical mixture of known mass may be measured 
by enclosing the mixture in a thermally isolated calorimetric “bomb” of high thermal conductivity, 
igniting the mixture by an electric spark, and measuring the resultant temperature rise of the enclosure, 
whose thermal capacity is known. Systematic corrections must be made for the spark energy and 
for any imperfections in the thermal isolation. 

Example 5.11 b. The radiant flux from a surface of large area may be measured by optically imaging 
that surface upon an energy receiver at the focus of an appropriate optical system. The receiver 
may be a bolometer or a photocell so that the measurement actually made is a resistance change 
or an emission current. Rather than measuring and correcting for the transmission factor of the optics 
and the characteristics of the energy receiver, calibration is usually effected by viewing a black- 
body surface of the same area, held at a known temperature. 

Example 5.11c. The convective heat flux delivered by an impinging gas stream to a portion of 
a surface may be measured by replacing that portion with a sheet of material of known thermal 
conductivity and measuring the temperature gradient in the material in a direction normal to the surface. 

Example 5.11<f. The mass flow rate of a fluid in a pipe is measured by use of a venturi in which 
pressure differential Ap and upstream pressure p and temperature T are measured. The basic product 
K 0 (p*Ap/T ) 1/2 , where K 0 is a calibration factor for certain nominal (bogey) conditions, must be 
modified by systematic corrections which were indicated in Example 4.22d. 

Example 5.11c. The mass flow rate m of a fluid in a pipe is measured by a Thomas-type of 
calorimetric flowmeter. A grid of heating wires uniformly distributed across the cross section of 
the pipe delivers heat to the fluid at the rate W; two similar grids upstream and downstream of the 
heating grid act as resistance thermometer elements so that the temperature rise ATof the fluid may 
be determined. The assembly is thermally isolated so that the process is adiabatic. If c is the specific 
heat capacity of the fluid, 


me* AT = W 


A systematic correction is required for the variation of c with fluid temperature. The grids may 
not provide uniform heating or correct averaging of temperature over the cross section of the pipe, 
but a systematic correction may be correlated with N Re if an empirical calibration is performed. 

Example 5.11/ A liquid flows in a circular pipe of radius R in which there is fully developed 
axially symmetric flow. The average axially directed speed u is to be deduced from an ultrasonic 
flowmeter in which a sound pulse is sent between two transducers located at the cylindrical coordinates 
(-/?, 0, Z{) and (R, x, z\ + L). If t x is the transit time when the sound travels in the direction of 
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the fluid flow, and t 2 is the transit time when the sound travels in the opposite direction, the velocity 
u is to be deduced from the difference 


At = t 2 - h • 

If fluid density is constant everywhere and fluid velocity is very small compared to the speed of 
sound c A in the fluid, then the formula giving the relation between u and At is of the form 

u = Kc 2 A *At/(2L) 

where K depends on the velocity profile. For a completely flat profile (fluid velocity the same at 
every location), K = 1; for a fully developed laminar profile, K = 0.75; for a fully developed turbulent 
profile, K — 0.844. Thus, if the velocity profile is known to be laminar (A^, < 1000) or turbulent 
(N Re > 2000), systematic corrections can be applied. However, if the Reynolds number lies in the 
transition region or if the velocity profile is not known for any reason, the assumption K = 0.797 
would yield a fractional error not greater than 6 percent. The uncertainty may be larger if the profile 
is not fully developed or is distorted because of the piping configuration. 

5.12 i Profile determination by continuous traverse . An empirical determination of the profile by continuous 
traverse is usually desirable when a new instrument, measurement technique, or physical situation is first 
introduced. Once such a determination is made, subsequent repetitive measurements may be made by the 
simpler methods treated in Secs. 5.10 and 5.11, because the required calibration factor or coordinate usually 
called for by those methods has been established with greater certainty than is possible by purely theoretical 
analysis. 

An appropriate procedure is initially to perform continuous traverses over the range of independent 
parameters that are, a priori, believed to influence the profile, thereby to establish the appropriate correction 
factors for the range of parameters, and then to replace the traversing method with the simpler single- 
measurement method. If the independent parameters are nondimensional quantities, the amount of work 
is usually minimized, and the utility of the results is augmented. 

Because of its mechanical complexity, a continuous traverse is usually convenient only in the direction 
of that coordinate in which the measured variable’s profile is least predictable. For example, in measuring 
the profile of fluid flow in a straight length of pipe, axial symmetry is likely and a radial traverse is more 
important than a circular one. 

In the time that is taken to complete the traverse, it is necessary that the variable u being measured change 
less than the permissible uncertainty Au in that measurement. On the other hand, the traverse may not be 
made so speedily that the instrument used in the traverse cannot follow the changes in u with distance x. 
If the traversing-instrument system has time constant r (see Chs. 6 and 7) and the gradient of u is du/dx, 
then it is necessary that 

(5.12-1) dx/dt < (Au/t)/ ( du/dx) 

If (du/dx) varies appreciably over the distance of traverse, it is necessary either that 

(i) (du/dx)^ replace (du/dx) in Eq. (5.12-1), if dx/dt is constant, or 

(ii) dx/dt be varied so as to satisfy Eq. (5.12-1) over the entire range of traverse. 

5.13 Integration or averaging by discrete local measurements. When the spatial variation of a physical 
variable u can be described by an analytic formula that contains n independent constants, the average value 
u may be deduced from local measurements of u at n locations which depend on the form of the formula. 
The case n = 1 was treated in Sec. 5,10. When the spatial variation is not predictable, the approximate 
value of u may be estimated from measurements at n discrete locations. If only one space dimension X 
is involved, so that u = u(X), this estimation is equivalent to fitting a polynomial of degree n to u(A^). 
The extent to which any weighted sum of n measurements agrees with the integral of u(X) t over any stated 
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X-interval, depends on the choice of locations at which measurements of u(X) are made and on the weights 
assigned to each measurement. 

For any one coordinate X and any one interval 

x a < X < x„, 

four methods of selecting the measurement locations and of summing the measurements will be described: 

A. Equally weighted measurements at n equally spaced locations. (Centroid-of-equal-areas method). For 
any given n, this method is the simplest and least accurate of the four methods. 

B. Appropriately weighted measurements at n equally spaced locations. (Newton-Cotes method). This 
is the only one of the four methods which requires measurements at the extreme ends of the interval. 

C. Equally weighted measurements at n appropriately spaced locations. (Chebyshev method) 

D. Appropriately weighted measurements at n appropriately spaced locations. (Gauss method). For any 
given n, this method is the most accurate of the four methods. 

Table 5-1 9 lists locations and weights for various values of n and for the following conditions: 

(a) In rectangular coordinates, the range 


X a <X<X b 


has been normalized to 


0 < x < 1 


by the transformation 


* = (X- X a )f(X b ~X a ) 


(b) In polar coordinates, the range 


0 < R < R m 


has been normalized to 


0 < r < 1 


by the transformation 


r = R/R m 


[Note N5.13.1] 

The number n must not be so large that the devices used to make the measurements significantly affect 
the distribution w(X) or its spatial integral or its spatial average, unless these devices are a permanent part 
of the mechanical installation and exert a permanent influence on the profile of u. [Note N5.13.2J. 

5.14 Average value of a function of several measured variables. If a quantity z is computed from 

measurements of several variables u 2 Uj ... that vary with some parametric variable * that represents 

spatial location or time, so that 


’For ease of later reference, Table 5-1 appears at the end of this text on page 220. An earlier version of this table was prepared 
by the author for the ASME Power Test Code on Velocity Measurement, and appears also in Ref. 4-6. 
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z = z(u,.u 2 Uj....) 

Uj = Uj(x), 


(5.14-1) 

(5.14-2) 


the technique for computing the space-averaged or time-averaged value Zg of z over the interval x a < x < x b 
depends on whether the space variation or time variation of the measured u ; is random or systematic. Here, 
Zo may be considered as being given by 


(5.14-3) ( x b - x a )»zo 


■ r- 


The variable x may represent a space variable like distance, area, or volume or a time variable like time 
or phase angle, 

A. Random variation . If all of the measured variables vary randomly with respect to x, so that there 
is no correlation between the local or simultaneous values of any pair of the Uj, then, over the interval 
x a < x < x b , each w, may be averaged to yield ujq given by 

(5.14-4) (x b - x a )*Ujo = I UjWdx , 

^ X a 

and the average value of z computed as 


(5.14-5) zq — z(U]q,U2o 

This procedure usually is the simplest, because it involves a minimum of instrumentation, computation, 
and data processing. Its applicability becomes more likely as the interval (x b — x a ) becomes smaller. 

B. Systematic variation . If the u } vary systematically with respect to space or time, so that there exist 
correlations among the local or instantaneous values of the Uj , it is necessary first to compute the local or 
instantaneous value of z, as given by Eq. (5.14-1), and then to compute Zq by Eq. (5.14-3). This procedure 
is usually complex because it requires a sufficiently large number of computations of z by Eq. (5.14-1) 
to permit adequately accurate computation of the integral in Eq. (5.14-3). 

C. Combined random and systematic variations. An intermediate level of complexity in instrumentation 

is achieved if some of the variables, say u x to u k , are uncorrelated with each other or the other Uj, and 
only the variables are correlated with each other. Then Eq. (5.14-4) is used to find a, 0 ,«20»- 

and these values are inserted into Eq. (5.14-1) to give 

(5.14-6) z = Z(«io >«20 ) 

This computation may be easier than one in which all Uj vary with x, but the integration (5.14-3) must 
still be performed. 


5.15 Error due to incorrect averaging. Since the averaging operation represented by Eq. (5.14-5) generally 
is much less complex than the operation represented by Eq. (5.14-3), the pragmatic question arises of when 
the simpler operation is an adequate substitute for the more complex one. If the measured quantities vary 
systematically with x, the error Azq that is made by treating them as random is given by 


(5.15-1) Azo — z(wt0»«20»- *• >**/)»••• »«no) 


r 

x b - x a J, 


z(u u u 2 ,...,Uj u„ ) dx 


and the fractional error, if small, is approximately Azo/z(«| 0 ,« 20 . - ) • 

The magnitude of the error may be estimated by assuming a simple analytic or graphic approximation 
of the functions uj(x), on the basis of some preliminary experiments, previous experience, or other physical 
intuition, and then performing, analytically or graphically, the computation represented by Eq. (5.15-1). 


no 



Special case A. In the particular case where the quantity z is the product of two variables, so that 
(5.15-2) z = u { u 2 , 


the error Azo is given more readily by the formula 

(% 

(5.15-3) (x b - x a )*Azo= - 1 (Wj - M,o) («2 - u 20 ) dx . 

^ x a 

In any case where the variables w, and u 2 vary randomly and there is no correlation between them, the 
integral in Eq. (5.15-3) approaches zero as the interval (x b — x a ) becomes larger. 

Special case B. In the particular case where the quantity z is the product of three variables, so that 

(5.15-4) z = u { u 2 u 3 

the error Azo defined by Eq. (5.15-1) is given more readily by the formula 
(5.15-5a) (jtfc — x fl ) # Azo = — ^Au^Au^Au^dx — u l0 ^Au 2 9 Au 3 *dx 

— u 20 ^Au] % Au^dx — w 30 ^AufAu-fdx 

where 

(5.15-5b) Au k = u k — u k o k = 1, 2, 3 

and all integrals are from x a to x b . 

Special case C. In the particular case where the quantity z is given by 
(5.15-6) z = U|H 2 /w 3 

the error may be estimable by the substitution « 3 = l/w 3 and the application of Eq. (5.15-5). 

This case occurs when the bulk velocity of a fluid in a duct is to be estimated from local measurements 
with a traversing pitot-static probe and a traversing temperature probe, or with rakes of such probes. 

Example 5.15a. The local linear velocity v and the local density p of a fluid flowing in a circular 
pipe of radius R are given by 


v = v r (l - r 2 /R 2 ) 

P = Pc( 1 + cxr 2 /R 2 ) 

where r is the radius at which v and p occur. 

The true mass flow rate through the pipe is 

f* 

m — \ vp m 2xrdr 

Jo 

The area-averaged velocity v and density p are given by 
ttR 2 v = l V2vr*dr 

Jo 

xR 2 p = l p*2xr*dr 

Jo 



so that the fractional error in assuming 


m = *R 2 xp [Note N5. 15.1] 


is 


a/(6 4* 3a) . 

If this error is to be less than 1, 2, 3, or 5 percent, a must be smaller than 0.06, 0.12, 0.19, or 
0.35, respectively. 

The velocity profile postulated in this example is that of fully developed laminar flow. 
Example SASb. If the velocity and density profiles in Example 5.15a were 

V = v c (l -x n ) [Note N5.15.2] 

P = P c ( \ + &x n ) 

where .r s r 2 /R 2 and r is the radius at which v and p occur, the fractional error in assuming 

m = tR 2 v p [Note N5.15.3.] 

would be 


[n/(2n + 1 )][a/(n + 1 + a)] 

Example 5.15c. The instantaneous sinusoidal current / through an impedance element and the 
instantaneous sinusoidal voltage drop e across the element are given by 

i = i m sin ( 2xtlT) 

e = e m sin[(2T t/T) - v?] 

where T is the period of the electrical oscillation. Average current, voltage, and power, as measured 
by integration over the time interval 




are 

/ = 2 ijv 
e = (2c m / tt) cos y? 
ei = ( l/2)/ m c m cos*? 

Hence the fractional error in using the product of the averages rather than the average of the products 
is 

(tt 2 /8) - 1 * 23 % . 

The error is independent of the phase angle as long as the integration interval is from t = 0 to t = 772. 
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Use of the product of rms averages rather than the product of arithmetic ones would yield the 
true average power. 

Example 5.15d. If each of three quantities u u u 2 , u 2 is expressible in terms of a normalized 
dimensionless parameter jc in the form 

Uj=aj[\ +{bj- 1)jc] (/ = 1^2,3) 


where 


0 < jc < 1; bj > 0; b } * 1 
and if the individual (arithmetic) averages are defined as 

Uj 0 = \uj dx 

(all integrals are from x = 0 to x = 1 ), 

then the fractional errors in estimating the functions 

fa — j(l/“3) dx; fb- j(«|/«3) dx; f c = j(M,M 2 /« 3 ) dx 
by means of the approximations 


fa = 1 /u 30 ; 


f b * u w /u }0 ; 


fc * “lo^zo/ u 30 


are given by 


A fa/fa = (« 3 / 2 )/(b 3 - 1 ) 

Aft//* = a 3 (Z > 3 -b, )/[((>, + l)(b 3 - l) 2 ] 

Af c /f c = 2a 3 (b, -b 3 )(b 2 -b 3 )/Ub { + 1 )(b 2 + 1 )(b 3 - l) 3 ] 

where 


a 3 = (6 3 + 1 ) In b 3 — 2 (b 3 — 1 ) 


[Note N5.15.4] 

lix represents the quantity sin (lirt/T), the formulas apply to time integration over a quarter period 
(77 4) of a sinusoidal wave. If x represents the quantity r 2 /R 2 , the formulas apply to integration over 
the cross section of a circular pipe of radius R. 


5.16 Nonlinear averaging. If instrument indication y represents the value x of some physical variable, 
and if y(*) is a continuous function, a periodic variation of x about some mean value x 0 will produce a 
corresponding mean indication y 0 . However, y 0 will not correctly represent y(Jt 0 ) if 

(a) the arithmetic mean of x is desired, but y is not a linear function of x; 

(b) the rms mean of x is desired, but y is not a linear function of x 2 ; 

( c ) the geometric mean of x is desired, but In y is not a linear function of In x. 

The difference between the indication x(y 0 ) and the true mean x 0 will depend on the magnitude and wave 
shape of the periodic variation; if the difference is sufficiently small, it may be neglected. A consideration 
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of the magnitude of the difference in certain simple or extreme cases may be sufficient to provide an intuition 
of whether some comparable actual situation warrants consideration of nonlinear effects. 

Case A. The variable of interest, x f varies sinusoidally with time, so that 

(5.16-1) x = x 0 [\ + b sin(27r//r)] 

but the measuring instrument is such that its indication y is given by 
(5.16-2) y = ax" . 

If b < < 1 , if averages are taken over an integral multiple of T or over a time interval very much larger 
than T, and if y 0 denotes the arithmetic average of y, then, 

(a) if x 0 denotes the arithmetic average of x , the fractional error in x 0 is 

(5.16-3) [x(y 0 ) - x 0 ]/x 0 * (n - \)b 2 /4 ; 

(b) if x 0 denotes the rms average of x, the fractional error in xl is 

(5.16-4) [x 2 (y 0 ) - xlyxl ~ (n - 2)b 2 /4 . 

Thus, 

(a) when n = 1 , there is negligible error in measuring the arithmetic average; 

(b) when n = 2, there is negligible error in measuring the rms average; 

(c) for any n, as long as the relation (5. 16-2) holds, there is negligible error in measuring the geometric 
average. 


Case B. If x has a rectangular waveform with period T, so that, if m is an integer. 


(5.16-5a) 

X rn X(t) = *o( 1 + b) 

for 

mT (m + 1/2 )T 

(5. 16— 5b) 

X *x(t) - x 0 ( 1 - b) 

for 

(m + 1/2 (m+ 1 )T 


and if the instrument is such that 
(5.16-2) y = ar n 


then, if y 0 denotes the arithmetic average of y over an integral number of cycles (or over a time interval 
that includes a very large number of cycles), the value of the fractional error 


[x(y 0 ) - xqI/xo 


is given by 



(5.16-6a) 

2 _<1/n, [(l + b) n + (1 - b) n ] (Un) - 1 


which may 

be approximated, for b ^ 0.8, by 


(5.16-6b) 

1.43b 2 /( 1 + 1.3b 2 ) if n = 4 

0.50b 2 /( 1 + 0.2b 2 ) if n = 2 

— 0.25b 2 (l + 0.4b 2 ) if/i = 1/2 
-0.37b 2 (l + 0.4b 2 ) ifn = 1/4 


Example 5,16a. In an orifice*type flowmeter, the measured differential pressure varies with the 
square of the velocity of flow. If there is a high-frequency fluctuation in this velocity, superposed 
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on the mean flow velocity, the differential pressure indicated by a manometer too slow to follow 
the fluctuation will correspond to a velocity higher than the arithmetic mean flow velocity. 

Example 5.16Z>. A constant-temperature hot-wire anemometer utilizes an electronic circuit to provide 
substantially instantaneous adjustment of wire current to maintain constant wire temperature; the 
wire current is measured. This instantaneous current is proportional to the fourth root of the 
instantaneous fluid velocity. If a rapid, periodic fluctuation is superposed on the steady flow velocity, 
without any flow reversal, the arithmetic mean of the current will represent a fluid velocity lower 
than the arithmetic mean fluid velocity. 10 

Example 5.16c. In a viscous fluid stream, in which there exists a constant gradient of velocity 
in a direction perpendicular to the flow velocity, velocity is deduced by measuring the force on a 
small disk (“impact disk”) whose plane is normal to the flow. The force on an element of area 
of the disk is proportional to the square of the velocity of the fluid impinging on that element of 
area. The measured force will correspond to a velocity higher than that existing at the center of 
the disk. However, the measured force will correctly correspond to the local impact force, local 
impact pressure, or local rate of momentum change at the center of the disk. 

Example 5.16d. A radiation pyrometer is sighted upon an area which is not of uniform temperature. 
Since the pyrometer’s indication is proportional to the fourth power of the temperature of the surface, 
the temperature corresponding to the indication will be higher than the arithmetic mean temperature 
of the area. However, the indication will correctly yield the arithmetic mean of the radiated energy. 

5.17 Energy or power transfer in measurements . The use of an instrument to measure a physical variable 
usually has one of two goals: 

(1) to determine the condition that exists while the instrument is installed, or 

(2) to determine the condition that would exist if the instrument were absent. 

In either case, one must consider the effect that the presence of the instrument has on the physical variable 
being measured. In the first case, the measurement is made because such effect is considered acceptable 
or can be made so by adequate and convenient control of the measurement conditions. In the second case, 
it is necessary to establish 

(a) that the presence of the instrument produces a negligible effect, or 

(b) that the quantitative effect of the instrument’s presence is known sufficiently well so that a 
systematic correction can be applied to the instrument’s indication in order to describe conditions 
after the instrument has been removed (any uncertainty in the knowledge of this correction 
becomes a random error); or 

(c) that the measurement conditions can be adjusted, and any auxiliary measurements made, so 
that situation (b) can be realized. 

The determination of the extent to which the presence of an instrument affects the physical variable being 
measured can usually be made by considering the ratio of the energy taken from the source by the instrument 
to the energy stored in the source; when the measurement is independent of the time interval during which 
the instrument is installed, the word “power” may replace the word “energy”. 

In practical situations, it is often adequate and more convenient to deal with some physical variable other 
than energy or power. 

Example 5.17a. An ammeter is installed in an electrical circuit. If the ammeter is later to be 
removed, its resistance must be known, because the absence of this resistance may alter the current 
in the circuit. [Note N5.17] 


10 Many practical constant-temperature hot-wire anemometers utilize a linearizing circuit that yields an electrical signal proportional 
to the fourth root of the wire current; this signal, rather than the wire current, is measured. Thereby, the correct average of a fluctuating 
signal can be obtained. 
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Example 5.11b. A voltmeter is used to measure the voltage across a resistor. If the resistance 
of the resistor is not negligible compared to the voltmeter resistance, the voltage across the resistor 
will change after the voltmeter is removed. 

Example 5.17c. The temperature of an object with low thermal capacitance (mass x specific heat) 
cannot be measured correctly with a thermocouple unless the wires leading from the junction lie 
along isotherms to prevent conduction of heat along the wires; or unless a systematic correction 
is made for such heat conduction. 

Example 5. lid. A turbine-type or orifice-type flowmeter in a pipe introduces a flow impedance, 
so that the flow rate would be different in the absence of the flowmeter. At any given fluid velocity, 
the effect is proportional to the ratio of pressure drop across the flowmeter to the pressure differential 
available to produce the pipe flow. (The product of volumetric velocity and pressure represents power.) 
However, there is no error in flow measurement, due to this pressure drop, if the flowmeter remains 
in the pipe as a permanent part of the installation. 

Example 5. lie. The presence of a traversing pitot tube, impact-plate flowmeter, or propeller meter 
to determine the distribution of local flow velocity may affect the area or the pressure differential 
available for the flow; thereby, the relation between local velocity and bulk velocity may be affected. 

Example 5.17 f. An ion gauge is attached to an enclosed high-vacuum chamber in order to measure 
gas density or gas pressure in that chamber. This gauge may act as a source or sink of gas, so that 
the gas density when the gauge is on is not the same as the gas density when the gauge is off. 

If an ion gauge is used to measure the pressure in a plenum through which there is a continuous 
flow of gas from an external source, the error will be small if the volumetric rate of flow of this 
gas is high compared to the (positive or negative) volumetric pumping speed of the gauge. (The 
product of volumetric flow rate and pressure represents power.) 11 

Example 5.17 g. A strain gauge is attached to the surface of a thin sheet in order to measure the 
elongation of the sheet. If the force required to operate the strain gauge in order to produce a given 
elongation is not negligible compared to the force required to produce the same elongation in the 
sheet alone, there will be an error in the determination of the force-strain relation that would exist 
in the absence of the strain gauge. (The product of force and elongation represents energy.) 


5. 18 Power transfer in null measurements . One of the methods of realizing the condition that the presence 
of the instrument shall have negligible effect on the physical quantity being measured (condition (a) of 
Sec. 5. 17) is to use a separate external source to provide most of the power that is required by the instrument 
to make the measurement. An effective technique is that of null measurement; it has the following important 
features: 

(1) It uses a detector that need not be accurate 

(2) It permits use of a detector with very high sensitivity— one whose sensibility (item (5) of Sec. 5.0) 
is much smaller than the probable error of measurement one is seeking to achieve 

(3) It still requires that the source being measured provide a slight amount of power (or else an 
off-null condition could not be detected), even though this amount is a small fraction of what 
would otherwise be required to effect a measurement. 

The null-measurement technique is illustrated by the following case examples: 


n The effect of pumping by an ion gauge may be determined (and used as a systematic correction if the effect is large enough) 
by having a second gauge attached to the chamber or plenum, and observing the change in indication of this second gauge when the 
first gauge is turned on or off. 
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{a} Voltmeter 

(b) Null potentiometer 

Figure 5 IK I.— Methods of voltage measurement. 


Case A. Voltage measurement . In Fig. 5.18.1(a) the emf E x generated by a source whose internal 
resistance is R x is to be measured with a voltmeter whose resistance is R y . When the voltmeter is connected, 
the voltmeter indication is 


E y - EJly/ (R x + Ry) 

rather than E x . The error may be eliminated if the voltmeter is replaced by the potentiometer shown in 
Fig. 5.18.1(b). A potential difference equal to E x is created by passing a controlled current I y through a 
resistor R y The current is derived from an external source. Either I y or R y is adjusted until I y R y = E x . This 
equality is indicated by zero deflection of the null indicator, whose resistance is R$. Both I y and R y are 
known very accurately, or else the product J y R y is measured very accurately by comparison with an 
accurately known emf, like that of a standard cell. (See, also, Examples 1.4a and 1.4b). The result does 
not depend on R x or R$ or on the null -detector’s calibration. 12 



(a) Ammeler 

(b) Null ammeter 

Figure 5. 18.2. — Methods of current measurement. 


Case B . Current measurement. In Fig. 5.18.2(a), the current l x from a source with shunt resistance R x 
is to be measured by inserting an ammeter of resistance R y between two terminals that would otherwise 
be connected to each other (short-circuited). If I x is the current that existed in this short circuit before the 
ammeter was inserted, the current indicated by the ammeter is 

1 X R X /(R X + R y ) 

rather than I x . The error may be eliminated if the ammeter is replaced by the circuit shown in 
Fig. 5.18.2(b). A current I y derived from an external source is adjusted until the null indicator, whose 
resistance is Rq, indicates that no current is passing through it. Then I x = I y . The current I y is measured 
very accurately. The result does not depend on R x or R$ or on the null-detector’s calibration. 13 


,2 The representation of a voltage source as a series combination of an open-circuit emf and a series resistance is Thevenin’s theorem. 
13 The representation of a current source in terms of a current and a shunt resistance is Norton’s theorem. 
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balancing pressure 

Figure 5.18.3— Null method of pressure measurement. 


Case C. Pressure measurement. In Fig. 5.18.3, a flexible metallic diaphragm separates two chambers. 
The unknown pressure, applied to one chamber, holds the diaphragm against a mechanical stop in the other 
chamber. This stop is electrically insulated from the remainder of the assembly. The pressure in the chamber 
containing the stop is increased until electrical contact is broken; the pressure at this instant is measured 
accurately and is equal to the unknown pressure. The accuracy does not depend on the elastic qualities of 
the diaphragm. [Notes N5.18.1 and N5.18.2] 

Case D. Temperature measurement. The correct value of the temperature T x of a calorimeter is achieved 
only if there is no heat exchange between the calorimeter and its surroundings. This condition is achieved 
by surrounding the calorimeter with an isothermal enclosure whose temperature is separately controlled 
to be equal to that of the calorimeter. The temperature difference may be measured with a differential 
thermocouple of fine wires. When this difference is zero, the temperature of the enclosure may be measured 
with an accurate thermometer, however bulky. The measurement does not depend on the quality of the 
differential thermocouple, or on the emittance of the surfaces involved. 


In similar fashion, a bolometer element designed to measure radiant energy may be surrounded by a 
separately heated isothermal enclosure whose temperature is kept equal to that of the bolometer. 


In all null measurements, the relation between off-null indication y and the physical variable of interest 
x can be established only by drawing some power from the source, even though this power is only a small 
fraction of the power that would have been drawn in a non-null measurement. The reason for this requirement 
is that the knowledge of the existence of a balanced or null condition requires the ability to detect when 
an unbalance exists. Thus, if (Ax)^ represents the limit of error that is acceptable in the measurement 
of x and if (Ay)^ represents the sensibility of off-null indication (the smallest off-null magnitude that can 
be reliably detected), it is necessary that, at and near balance, 

(5.18-1) (Ay) min <; (AxU . 

The derivative dy/dx is the sensitivity of off-null detection. This derivative is a maximum when, at an 
unbalance At, the power developed in the source is equal to the power developed in the detector. In electrical 
circuits, this equality is achieved when certain relations exist between source impedance and null-indicator 
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circuit impedance. In mechanical or thermal situations, it may be more difficult to define the requirements 
explicitly, but Eq. (5.18-1) remains dominant. 

The cases just treated will serve as examples. 

Cases A and B. For a given unbalance Ax (in voltage or current to be measured), maximum voltage will 
appear at the detector terminals if R 0 is as high as possible; maximum current will pass through the detector 
if R 0 is as low as possible; maximum power will be delivered to the detector if 

(i) in a dc circuit, R 0 = R x + R x 

(ii) in an ac circuit, where a resistance R is replaced by an impedance 

z = R + jX, 


it is necessary that 


- X x 4- R y 

and that 

Xo= - (X x + X y ) 

In ac .circuits, the matching of source and detector impedances is facilitated by the use of transformers. 
The use of a transformer that provides maximum power transfer will yield the best sensibility of off-null 
detection. 

It should be noted that, when maximum power transfer is desired, no serious loss of power results when 
there is a moderate deviation from unity in the ratio 


Rq/ (R x + R y ) 


For a ratio of 2: 1 or 3: 1 , the power delivered to the detector is only 12 percent or 30 percent less, respectively, 
than the maximum possible value. 14 

Case C. It is necessary that the diaphragm be so flexible that the pressure differential required to produce 
a clear separation between contacts is only a small fraction of the desired sensibility of pressure measurement. 
At the same time, the mechanical design must be such that the diaphragm is not damaged when full-scale 
pressure acts to hold the diaphragm against the stop. 

Case D. Not only must the sensibility of off-null detection be much smaller than the desired sensibility 
of temperature measurement, but the uniformity of temperature of the surface of the enclosure must also 
be adequate; auxiliary differential-temperature measurements may be used to confirm this adequacy or to 
provide a closer estimate of the mean enclosure temperature. 

5.19 Ultimate sensibility of measurements. Noise , The smallest change that is reliably detectable in any 
observation or measurement can be represented as the rms value of a random error in that measurement. 
For psychophysical observations, this value is determined by the abilities of a human observer; quantitative 
descriptions, based on statistical analysis of the abilities of a representative sample of persons, can be presented 
for an average observer, sometimes designated as the “standard observer”, although the sample is often 
chosen from a group of physically Fit young persons. 

For measurements of physical variables, the limit of sensibility is determined by the randomness of the 
motion of molecules, atoms, or charged particles or of the randomness of occurrence of events involving 
these particles. The magnitude of sensibility can be reduced by averaging the observation over a longer 
period of time; the magnitude usually is reduced in proportion to the square root of the averaging-time 
interval At. 


,4 This result derives from the properties of the maximum or minimum: a large change in the abscissa produces only a small change 
in the ordinate. This effect entered also into the discussions of Secs. 3.28 and 4.10. 
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In electrical measurements, it has become customary to consider that the quantity has a frequency spectrum 
and that averaging is the process of measurement over a frequency interval Af This quantity is on the order 
of magnitude of 1/(2 A/) but not necessarily equal to it. The advantage of using Af is that many steady -state 
electrical measurements represent the average (usually rms average) value of a signal that is fluctuating 
at a frequency near to f; Af is then an interval centered around / 15 

The term noise has been affixed to the rms value of the random fluctuations that ultimately limit the ability 
to make a measurement. When a detector is used to measure the output of a source of energy, both source 
and detector individually contribute noise. Furthermore, there may be several origins of noise in both source 
and detector. Each origin of noise may contribute a separately definable magnitude of noise. The resultant 
sensibility, that constitutes the limit of reliable detection, is the square root of the sum of the squares of 
all of these magnitudes, provided each of these contributions has the same dimensions. 

Some common limits of sensibility are listed below. (When they appear in the formulas given in this 
section, T is absolute temperature and k is Boltzmann’s constant.) 16 

A. Psychophysical optical observations . 

(1) The ability to detect the coincidence of two index lines is termed the vernier acuity of the eye. 
It is of interest in optical range finding and in dimension measurement by optical projection. It is on the 
order of 12 seconds of arc for a person with excellent vision. For the unaided eye at normal viewing distance, 
the vernier acuity becomes 0.015 mm; this distance is reduced in proportion to any linear magnification 
that is used. 

(2) The ability of the eye to detect the separation between two lines is termed the visual acuity of 
the eye. It is on the order of 1 minute of arc for a person with excellent vision. For the unaided eye at 
normal viewing distance, the visual acuity becomes 0.07 mm; it is reduced in proportion to any linear 
magnification that is used. 

(3) The ability of the unaided eye to detect the presence of an appropriately illuminated scratch on 
a polished surface is due to diffraction by the scratch. A 1 pm scratch, subtending a viewing angle of 1 second, 
is detectable. The observation provides information only on scratch presence— not on scratch width. 

(4) The eye can discriminate between two levels of luminance that differ by 2 percent, if the eye 
has been fully adapted to the luminance level. 

For example, application of this fact to the manually balanced disappearing-filament optical pyrometer 
indicates an ability to achieve balance to 1 K at a temperature of 1000 K. 

B. Mechanical measurements . The ability to make mechanical measurements of position is limited by 
Brownian motion and is rarely of interest in industrial or routine laboratory applications, but does enter 
into some highly sensitive measurements in research or standards laboratories. 

(1) When particles are suspended in a stagnant liquid, and a particle's position is averaged over a 
time interval At, the rms uncertainty fix in position x is given by 

(5.19-1) (fix) 2 = 2D*Ax 

where D is the diffusion coefficient of the liquid. [Note N5.19.1] 

(2) When the linear deflection x of a spring, whose spring constant is K x (force per unit deflection), 
is to be determined, the rms value of the uncertainty 6x is given by 

(5.19-2) (fix) 2 =kT/(2K x ) . 

(3) When the rotation tp of a torsion pendulum, whose torsional spring constant is (torque per 
unit angle of twist), is to be determined, the rms value of the uncertainty 8<p is given by 


19 The precise definition of Af must be made by the user; a common definition uses the ‘ 'bandwidth at half maximum” of a bandpass 
filter. (See Sec. 7.20.) 

,6 At 300 K, kT » 4 x 10- 21 J. 
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(5.19-3) (M 2 =kT/(2K,) 

C. Measurement of radiant energy. 

(1) The rms value bP of the random fluctuations of total radiant power P emitted by a radiator of 
area A and total emittance e is given by 

(5.19-4) (6P) 2 = 8eAokT 5 *Af 

where o is the Stefan-Boltzmann constant. (Ref. 5-1, p. 59) 

(2) The rms value bT of the random fluctuations of temperature Tof a body, like a radiation receiver, 
whose heat capacity (thermal energy required to produce unit temperature rise of the body) is C b , is given by 

(5.19-5a) (bT) 2 = kT 2 /C b . (Ref. 5-1, p. 354) 

The rate at which the body can exchange energy with its surroundings is proportional to its heat transfer 
rate K b (thermal power it exchanges per unit temperature difference). The rms value bP of the fluctuation 
in this power, because of the fluctuation 8T> is given by 

(5 - 1 9— 5b) (bP) 2 = ( K b *bT) 2 = 4kK b T 2 *Af . (Ref. 5-1, p. 356) 

(3) When the values of bP for both emitter and receiver are of significant magnitudes, the values 
of (5F) 2 given by Eqs. (5.19-4) and (5.19-5b) must be summed, after appropriate weighting for the solid 
angle subtended by the receiver, to obtain the total noise due to these causes. 

(4) For a laser operating as a single-frequency oscillator far above threshold, the rms variation bn 
in the number n of quanta emitted in any given time is Vn. (RefT 5-2, p. 105) 

D. Measurement of electrical quantities . 

(1) Thermal noise (Johnson noise, Nyquist noise) is usually the principal type of noise found in electrical 
components. [Note N5.19.2] It appears as an emf across the terminals of a resistive element even when 
the element is open-circuited and even when no current is passing through the element. When thermal 
equilibrium exists, the rms value be of the random fluctuations in the emf e that appears at the open-circuited 
terminals of a resistance R is given by 

(5. 19-6a) (be) 2 = AkTR'Af . 

(At 300 K, the noise power is 1.7xlO“ 20 »A/ watt if Af is in hertz.) 17 

(a) If the resistor is connected to a capacitance C and if the frequency /around which the interval 
Af is centered is such that 

(5. 19-6b) />> 1/(27 tRC) 

then 

(5.19-6c) (be) 2 = kT/C 

because the capacitor acts as an integrator of the noise. (At 300 K, the noise energy is 
2 x 10 “ 21 J.) 

(b) If the resistor is connected to a critically damped galvanometer (Sec. 6.3) whose natural 
frequency is /„, then the integration of noise is such that 


l7 The measurement of noise power in a wire resistor has been proposed as a means of temperature measurement (the noise 
thermometer ) because the power generated depends only on the temperature and on Af. 
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(5. 1 9— 6d) (be) 2 = itkTRf„/2 . 

(2) Current noise represents a random fluctuation Si in the current / passing through an electrical 
component. The magnitude of the fluctuation is proportional to some power of the current /. There are 
several possible sources, distinguished by their nature or their behavior. 

(i) Shot noise [Note N5.19.3] 

(ii) Generation-recombination noise (G-R noise) [Note N5.19.4] 

(iii) Flicker noise (1 If noise) is such that noise power varies inversely with frequency. Its cause 
is not well understood; its magnitude appears to depend on surface conditions. 

(3) When several sources contribute noise, the total noise voltage or current that appears at any pair 
of terminals is the square root of the sum of the squares of the individual noise voltages or currents. This 
quantity is sometimes described as the root sum square (rss) voltage or current. (The summation may be 
made only of quantities having the same dimensions.) The total noise power generated is merely the sum 
of the individual noise powers. 

Example 5 . 19 . If, in order to achieve maximum power transfer (Sec. 5.18), a load resistor at 
temperature T 2 is connected to a source resistor of equal value but at temperature 7j , then the noise 
power in each resistor is k(T t + T 2 )*Af. (If T\ = T 2 = 300 K, the noise power in each resistor is 
0.8 x 10 ~ 20 * Af watt) 

(4) When a device has more than two terminals or when there is a chain of devices, it is convenient 
to state the equivalent input noise— the noise at the input terminals that would create the same noise at a 
designated pair of output terminals if all succeeding components of the chain were noise free. This equivalent 
input noise may be described in one of several ways: 

(i) by stating the equivalent input noise power, as described above 

(ii) by stating the equivalent noise resistance R„, the value of a resistor whose thermal noise emf, 
at the temperature T of the sources (if all sources are at the same temperature) or at an 
arbitrarily specified temperature T, is the same as the total noise emf 

(5 . 1 9-7) R n = [total (be ) 2 ]/(4kT»Af) 

The power fluctuation 8P associated with a voltage fluctuation be across a resistor R„ or 
with a fluctuation Si in the current through that resistor is given by 

(5.19-8) bP = (be) 2 /R n = (bi) 2 *R n . 

(iii) by stating the value of the equivalent input be or bi 

(iv) by stating the equivalent noise temperature, T„, the temperature of a resistor whose thermal 
noise power would be equal to the total noise power 

(5.19-9) T n = (total noise power)/(4£»A/) 

(v) by stating the noise factor, F, the ratio of the noise power that actually appears at the designated 
output terminals of the device, to the noise power that would appear at the output if the only 
noise source were an ideal (noninductive and noncapacitive) wire resistor whose value was 
equal to the resistance of the source that is connected to the input terminals of the device. 
Thus, one may speak of the noise factor of a semiconductor, of an amplifier, or of a 
semiconductor connected to an amplifier. 


(5.19-10) 


actual noise power output 
noise power output from ideal source 
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This ratio may be expressed in decibels as (10 log F)dB. 

The terms noise factor and noise figure have been used by some writers to denote the power 
ratio itself or its value in decibels, but there has been no consistency in the allocation of names. 
Current U.S. practice is to use the terms interchangeably, regardless of what (nondimensional) 
unit is used. A further standard practice is to specify that, when Fis used in a product specifi- 
cation to characterize the performance of a device, the temperature of the ideal resistor shall 
be presumed to be 290 K, and the bandwidth shall be presumed to be 1 Hz. 

Note N5.19.5 gives some empirical information on noise sources and magnitudes. 

5.20 Signal-to-noise ratio. The indication of an instrument intended to measure some physical variable 
is impaired by the presence of random errors in that indication. The indication that would be obtained in 
the absence of such errors may be termed the “signal”; the indication that would be obtained in the presence 
of only the random errors may be termed the “noise”. An effective way to determine whether noise introduces 
a significant uncertainty in the measurement of a signal is to compare the signal power S (the power available, 
at the point of measurement, from the source being measured) with the noise power N available at the same 
point of measurement. The signal-to-noise power ratio (S/N) is sometimes stated in dB (Eq. 2.8-la). 

The noise power N may be considered to originate from the sources treated in Sec. 5.19, from random 
fluctuations in the variable being measured, or from randomly fluctuating external physical variables that 
produce corresponding fluctuations in the signal at the point of measurement. On occasion, systematic 
fluctuations have been included in the definition of noise as a matter of convenience; for example, power- 
line hum when its evaluation, correction, or cancellation is inconvenient. On such occasions, it is necessary 
that an author define what is included in N. 

There remains the question of how acceptable is a given ratio S/N. The distributions of the numerous 
random events that cumulatively contribute to the noise are sufficiently Gaussian, and the central limit theorem 
(Sec. 3.29) is sufficiently effective so that the Gaussian distribution (Sec. 3.18) may be considered to apply 
to the noise. The total rms noise amplitude (voltage, current, displacement, etc.) then corresponds to the 
quantity o in Eq. (3.18-1) and to the abscissa e a in Fig. 3.18(a). 

Loosely speaking, one may say that, for noise with a Gaussian amplitude distribution, if the signal amplitude 
is 2 or 3 times the noise amplitude (S/N = 4 or 9, respectively), the odds are about 1 :20 or 1 :400, respectively, 
that the signal will be obscured by the noise (Ref. 5-3, p. 22). 18 

5.21 Improvement of signal-to-noise ratio. If /V represents only random noise, the value of S/N may 
be increased by increasing the time interval At in which a reading is averaged or by narrowing the frequency 
band A f within which a signal is being accepted. Mechanical means of doing this are use of an instrument 
with low natural frequency or long time constant (Chapter 6). Electrical means are 

(a) use of analog integration 

(b) use of an integrating digital meter, such as one that converts voltage or current to pulse frequency, 
so that the pulses may be counted over a long period of time 

(c) use of synchronous modulation and demodulation (e.g., with electromechanical, photoelectric, 
or solid-state-circuit choppers) with a long-time-constant detector; if the original noise lay in a 
frequency interval Af and the detector time constant or the integration time is r, the factor by 
which the original signal-to-noise ratio is improved is on the order of V (r*A/) 

(d) use of narrow-bandpass filters. 

The choice of an averaging-time interval or its equivalent always represents a balance between the amount 
of noise reduction and the ability to follow fluctuations in the variable being measured. 

Other methods of increasing the signal-to-noise ratio are 

(a) reduction of the temperature of the principal sources of noise that are temperature dependent 
[Note N5.21], 

(b) use of null-potentiometer techniques to reduce current in a source of noise that is current dependent. 


18 Note that Ref. 5-3 uses S/N to denote the square root of the power ratio. 
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5.22 Instrument standards. In any one laboratory, instruments in daily use are occasionally checked by 
comparison with “working standards;” these, in turn, are occasionally checked by comparison with 
instruments which are “primary standards” for that laboratory. The primary standards may occasionally 
be checked by comparison with standards maintained by a national or regional standards organization, but 
such comparison may be rare and is usually inconvenient because great care may be required in transportation 
of the laboratory’s primary standard to another location, and considerable recovery and stabilization time 
may be required after such transportation before a reliable comparison can be made. 

The laboratory’s own primary standards should be maintained under conditions that ensure virtual certainty 
that they have remained unchanged since the comparison with the national or regional standards laboratory. 
Some means for such assurance are 

(1) The standards are in a room in which temperature and humidity are controlled, in which vibration 
and acoustic noise are minimal, and in which temperature and humidity are unlikely to change appreciably 
when there is a temporary failure of electric power or other utility service. A windowless basement room, 
well away from noisy, vibrating machinery, is often a satisfactory location. 

(2) Personnel who use these standards are few in number, permanent, professionally trained, and 
temperamentally suited. The professional training must include an understanding of the instruments and 
the techniques of using them, the ability to recognize and correct for systematic errors, and the ability to 
recognize and estimate random ones. Temperamentally, the personnel must be willing to attend carefully 
to all the numerous details usually required, and must have the integrity to admit any inadvertent mistake 
that may have compromised the accuracy of a standard; at the same time, they must have been assured, 
in advance, that such inadvertence would not be penalized. 

(3) The packaging and transportation of the laboratory’s primary standard to and from the national 
or regional standards laboratory must be performed with sufficient care to ensure that safe limits of 
temperature, acceleration, and vibration have not been exceeded. 

5.23 Criteria for instrument selection . The selection of an instrument for a particular application requires 
a human judgment that is based on the needs of that application and the characteristics of the available 
instrumentation. Most of the instrument characteristics can usually be expressed in quantitative form; fewer 
of the needs can usually be so expressed. Once the human judgment is made, it may be possible to formulate 
the conclusion as the sum of a number of weighted numerical gauges of each item considered. The items 
and the weights will vary with each application. Any formula for selection is the result of a human judgment; 
it is not a substitute for such judgment. 

The magnitude of systematic and random errors of an instrument, as treated in this chapter and further 
treated in following chapters, will depend on the range of circumstances under which the instrument will 
be expected to operate satisfactorily. The weight (importance) assigned to these errors will depend on the 
measurement accuracy required and on other performance factors, like those outlined below. In gauging 
the merits of any instrument for any given application, caution must be exercised not to condemn it over 
another merely because more is known of its errors than is known of the errors of the other, nor to appraise 
one instrument highly merely because less is known of its errors than is known of the errors of others. 

One of the more common compromises required is between speed and accuracy. The subject of dynamic 
response will be treated in the chapters that follow, but even when a so-called static measurement is made, 
the time required to make the measurement usually must not be excessive, even though random errors are 
always reduced when the integration interval is increased. 

Some other significant criteria for selection of an instrument or a measurement technique are listed here. 

(1) Installation and integration: quality and cost of personnel required to make the initial installation 
and to integrate it into the complete system or operation. 

(2) Maintenance required: frequency, quality, and cost of the work; quality, permanence, and cost 
of the personnel required and of their instruction. 

(3) Reliability: probability, severity, and consequences of malfunction. 

(4) Serviceability: availability of parts and of repair service over the life of the instrument; quality 
and cost of personnel required for such service. (The cost of available parts is usually a trivial fraction 
of other costs.) 

[Note N5.23] 
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NOTES FOR CHAPTER 5 

N5.1. In this example, or whenever a pointer-and-scale instrument is used to provide a quick indication 
of a momentary condition (like the maximum value of a momentary pulse), it is important that the indication 
be quickly perceptible. A broad, spade-type pointer and a coarsely graduated open scale (Fig. 5.1) that 
relies on visual interpolation between scale divisions is more useful than a finely divided scale or a knife- 
edge pointer. 



Figure 5. 1 — Pointer and scale 
design lor quick perception 


N5.4 The deviation of the sea level value from the standard value g n (Table 2-1) is less than 0.3 percent, 
as reported in the Smithsonian Physical Tables for over 100 locations, worldwide. The reduction with altitude 
is 0.03 percent per kilometer. 

N5.6.1 If separation of these effects is of interest, the following procedures may be helpful: 

(i) when backlash is negligible, elastic hysteresis in the presence of dry friction can be evaluated 
by performing calibrations in the presence of mild vibration. In an electrobalance, dither (a 
superposed alternating current of relatively small amplitude) may be used. 

(ii) when dry friction is negligible, elastic hysteresis in the presence of backlash can be evaluated 
by performing a calibration in one direction without ever overshooting a calibration point and 
then performing a calibration in the opposite direction by overshooting each calibration point and 
then returning to the desired input. The amount of overshoot should be slightly larger than the 
backlash. 

(iii) when both backlash and friction are significant, they generally mask the elastic hysteresis. 

N5.6.2 The magnitude of the hysteresis and drift is approximately proportional to the magnitude of the 
change in spring deflection that produced it. The values of hysteresis and drift will therefore be largest 
when the input change represents the full span of the instrument. 

N5.6.3 The drift generally occurs exponentially with time, so that the “final” indication is the asymptote 
of the curve of indication vs time. 

N5.6.4 The temperature coefficient of the modulus of elasticity of NiSpan C, a high-quality spring material, 
is about -2 x 10" 5 /°C (it varies with the heat-treatment temperature); that of ordinary high-quality spring 
steel is about -20xl0“ 5 /°C. The coefficient of linear thermal expansion of both materials is about 
1 x 10" 5 /°C. 

N5.7.1 The net thermal emf to be expected at the terminals of an instrument are 
3 to 10 /iV in high-quality industrial instruments 
1 to 3 /iV in high-quality instruments for routine laboratory use 
0. 1 to 1 jxV in highest-quality laboratory standard instruments 

N5.7.2 In some applications, manipulative techniques may permit reducing the effect of thermal emf s by 
a factor of 10 or 100. For example, in using a Wheatstone bridge, reversal of power-supply polarity, leaving 
all other wiring untouched, permits averaging of the null-detector indications to yield the correct off-null 
reading. 
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In other applications, one may seek to create an isothermal environment for the low-level circuitry by 

(i) mounting positive and negative terminals next to each other on a massive copper block with thin 
insulating bushings; 

(ii) surrounding the low-level circuitry with an isothermal enclosure; 

(iii) minimizing the amount of heat generated in the enclosure; 

(iv) using only massive all-copper switches, copper wire and terminals, manganin resistors, and 
low thermal emf solder. 

N5.7.3 The electric and magnetic effects of power-line hum may be reduced by some of the following 
expedients: 

(i) use of coaxial or triaxial cables and electrically insulated enclosures, whose shields are all 
connected together at one and only one terminal 

(ii) connecting this terminal to a point at the same ac potential as the ground or neutral line of the 
power supply 

(iii) filtering the power-line input to the measurement system to remove significant harmonics 

(iv) if hum is of steady amplitude and phase, using a hum-bucking circuit that provides independent 
control of amplitude and phase 

(v) twisting of wire pairs that carry the same signal in opposite directions 

(vi) shielding these pairs with material of high magnetic permeability 

(vii) using power supply transformers of balanced construction, with separately shielded primary and 
secondary windings (Interwinding capacitance is usually less than 1 pF.) 

(viii) physical separation of low-level circuits from the power lines 

(ix) use of band-rejection filters, preceding the measuring instrument 

(x) measuring the average value of the signal over a sequence of time intervals, each of which is 
exactly equal to the period of the hum, or to an integral number of such periods. The effect 
of hum is eliminated and higher-frequency noise is attenuated. An average of these averages 
will attenuate noise at frequencies below the hum frequency. (The successive intervals need not 
be contiguous, so that switching operations may be interposed between these intervals.) Integrating 
digital voltmeters often use this method of hum rejection. 

N5.10.1 A laminar profile is to be expected if the Reynolds number N Re of the flow is less than 1000. The 
profile is then given by 


u/u c = 1 - r 2 /R 2 

where u c is the velocity at the center of the pipe (r = 0). 

If a flat profile exists at the entrance of a square-ended pipe, the profile can be considered fully developed 
at a distance 


x = 0. 2 RNfo 

downstream from the entrance of the pipe. 

N5.10.2 A turbulent profile is to be expected if the Reynolds number N Re of the flow is greater than 2000. 
The profile is then given by 


u/u c = (1 - r/R ) 1/7 

where u c is the velocity at the center of the pipe (r = 0). 

If a flat profile exists at the entrance of a square-ended pipe, the profile can be considered fully developed 
at a distance 
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x = 40 m Re 


downstream from the entrance of the pipe. 

N5.13.1. The tabulation of x-values in Table 5-1 may be used to determine the equivalent values for any 
other coordinate system, by the following procedure. 

(1) Let the limits of the new coordinate s be s a and s b so that 

s a <s<s b 

(2) Write the element of area dA in the new coordinate system as 


dA - B*ds 


(3) Then the transformation between s and x is given by 



Example 5.13a. In polar coordinates, if 

R a < r < R b 
dA = 2irr*dr, 


then 


*= {r 2 - R\) / (Rl - Rl) 

Example 5.13&. Fluid flowing in a pipe immediately downstream of an elbow may have a strong 
velocity gradient along a diameter lying in the plane of the elbow. To find bulk velocity, it may 
be best to take the element of area, as it is shown in Fig. 5. 13, at a distance r from the pipe’s center, 
so that 



Figure 5. 13 — Element of area in a circular pipe with diametral velocity 
gradient. Dots show locations for six-point integration. 
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—R < r < R 
dA — 2dr*\! (R 2 — r 2 ) 


Then 


x = (1/2) + (l/7r)[(r//?)V[l - (r 2 /R 2 )] + sin _l (r/K)] . 

Here, x = 0 when r = -R and x = 1 when r = R . 

However, if only a single measurement is made for each element of area, with the points of 
measurement lying on the pipe diameter, the computed bulk velocity will be too high. A closer 
approximation will be obtained if each element of area is assigned the average of two measurements 
made at locations which are on each side of the diametral line, by a distance equal to 0.6 \I (R 2 — r 2 ) 
(the 0. 2-0.8 rule). These locations are shown by dots in Fig. 5.13 for the case n = 3 and Chebyshev 
integration. 


N5.13.2 In the most general case where u varies along three orthogonal directions AT,, X 2 , X 3 , a different 
method of integration and a different number of locations n 2 , n 3 , respectively, may be selected for each 
of the directions, to suit the circumstances. 

Usually, methods using equally weighted measurements are preferred because then the simple sum of 
all measurements conveniently provides the desired integral of u. 

Example 5.13c. To measure bulk fluid velocity in a circular duct, the centroid-of-equal-areas method 
may be used with ri\ equal to 2, 3, or 4, for measurements around a circle at any given radius; 
and the Chebyshev method may be used with n 2 equal to 2, 3, or 4 for measurements along any 
given radius. If almost axially symmetric flow is expected, n, = 2 may be sufficient. If fully 
developed flow is expected, n 2 = 2 may be sufficient. If the flow is expected to be fully developed 
and almost axially symmetric, a total of four measurements may be adequate. 

Example 5.13d. The average speed v of a river is measured by four vertical arrays of current 
meters. The arrays are in a straight line normal to the direction of flow and, by the Chebyshev method, 
are located at 0.1027, 0.4062, 0.5938, and 0.8973 of the river width; each array consists of two 
current meters located at a distance below the water surface equal to 0.21 13 and 0.7887 of the local 
water depth. If this depth, at the four horizontal locations, is 4, 7, 8, and 5 m, respectively, and 

if the sum of each pair of current-meter readings is s,, s 2 , s 3 , and s 4 respectively, then the velocity 
v is given by 


48v = 4 j, + 7 s 2 + 8j 3 + 5 s 4 


N5.15.1 The integrals in Example 5.15a are 


m = *R 2 \ c p c ( 3 + a)/6 
v = \ c /2 
p = p c ( 2 A- a)/2 


N5.15.2 The progressive change in velocity profile as n decreases from an initially high value to an ultimate 
value of unity qualitatively resembles the development of a laminar-flow velocity profile when a fluid enters 
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a square -ended circular pipe from a substantially infinite plenum. The profile is initially almost square; 
its fully developed shape is represented by n = 1. For further details, see Notes N5.10.1 and N5.10.2. 

N5.15.3 The integrals in Example 5.15b are 

m — TR 2 v c p c [n/ (n 4 1 ) ][ I + a! (In 4 l)] 

v = \ c m n! (n 4 1 ) 

p = p c (n 4- 1 +a)/(/i+ 1 ) 

N5.15.4 A common example in gas dynamics is the measurement of the mass flow rate through a restriction 
(such as the nozzle supplying air to a jet engine under test). The square of this flow rate is proportional 
to p 9 Ap/T, where p is upstream pressure, A p is the pressure drop across the restriction, and T is upstream 
temperature. Then u { = p, u 2 = A p y i/ 3 = T. For practical reasons, one may choose to average separately 
the local temperature, or the local density (which is proportional to p/T), or the local mass flow rate. 

N5.17 If R s = source resistance 
R l = load resistance 
R a = ammeter resistance 

E p = acceptable probable fractional error of measurement 
then R a should be much smaller than ( R s 4 R L )/E p unless a systematic correction is made. 

N5.18.1 The method is useful when 

(a) the unknown pressure is that of a fluid to which a high-precision pressure gauge may not be safely 
connected. 

(b) there are many pressures to be measured. Each pressure source is connected to its own respective 
chamber-diaphragm assembly, but all chambers containing stops are connected to a common source 
and to a single high-precision pressure gauge. As pressure of the common source is increased 
at an accurately known linear rate, the time at which any individual contact is broken is a measure 
of the respective pressure. 

If any individual pressure is fluctuating, it must be connected to the chamber-diaphragm assembly through 
a capillary tube or other pneumatic resistance in order to provide appropriate averaging of the pressure 
(see Secs. 6.6 and 7. 12); otherwise, contact might be broken at a momentary extremum of the fluctuation. 

N5.18.2 A comparable mechanical arrangement has been used for acceleration measurement; the end of 
an acceleration-sensitive spring-mass combination (Sec. 6.1) is held against a stop by a calibrated spring. 

N5.19.1 This relation was deduced by A. Einstein in 1905. The diffusion coefficient is the mass of particles 
that diffuse through unit area in unit time per unit concentration gradient. (Concentration represents the 
mass of particles in unit volume.) 

N5.19.2 Thermal noise represents random fluctuations in the velocity of free electrons in a resistive element 
and results principally from collisions between the free electrons and the lattice atoms. 

N5.19.3 Shot noise represents fluctuations in the number of free electrons. 

N5.19.4 Generation-recombination noise represents fluctutations in the number of charge carriers and is 
said to result from the generation and recombination of carriers, like electrons and holes in semiconductors, 
although a more precise description is that the noise results from randomly occurring transitions between 
energy levels. 
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N5.19.5 In the following formulas, e, i, R f g, and Af are, respectively, in volts, amperes, ohms, siemens, 
and hertz. (Refs. 5-2 and 5-3; also see Footnote 18 on p. 123) 

(1) Shot noise in composition resistors (carbon, metallized film, or cermets) can be represented by 
a noise voltage be across the resistor given by 

(N5.19-1) \be\ = C] x 10 -8 e log 2 R 

where 


0.1 sq <10. 

(2) For a space-charge-limited thermionic diode, the random fluctuation Si in current is given by 

(N5.19-2) (6i) 2 = 3(1 - ir/4)kTg*Af 

where 

(N5.19-3) g = di/de 

is the transconductance at the operating point. 

(3) For a forward-biased semiconductor diode carrying small currents, the random voltage be that 
appears across the diode is given by 

(N5. 19-4) (be) 2 = 2kT*Af/g 

where g is the operating-point transconductance given by Eq. (N5.19-3). 

The noise is less than the thermal noise that might be expected from Eq. (5.19-6a) because a 
biased diode is not a system in thermal equilibrium. 

(4) For diodes in other than the ordinary operating region (e.g., for diodes in the reverse-bias or 
saturated region) the noise is so much greater in magnitude that the device is not used to detect 
small electrical signals. In fact, a reverse-biased diode is often used as a random-noise generator. 

(5) For a thermionic triode, R„ as defined by Eq. (5.19-7) is given by 

(N5.19-5) R n * 2.5/g 

where g is the transconductance given by Eq. (N5.19-3). 

Tetrodes and pentodes are about three times noisier than triodes. 

(6) For a bipolar transistor, each lead may be considered to have current noise in addition to any 
thermal noise developed in resistors that are in those leads. At moderate frequencies, each lead 
may be considered to have a random fluctuation Si due to shot noise and given by 

(N5.19-6) (Si) 2 = 2 qJ*Af 

where i is the current in the lead and q e is the electron’s charge (*1.6x 10“ 19 C) 

There may also be appreciable thermal noise (Eq. 3.19-6a) generated by the base resistance. At 
low frequencies, this noise may be augmented by flicker noise, so that the fluctuation bi b in base 
current i b is given by 
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(7) For a field-effect transistor (FET), the principal effect of the combination of thermal and flicker 
noise can be described by stating that a random voltage fluctuation be may be considered to have 
been injected into the gate lead, where 

(N5.19-8) (be) 2 = SkT{l +/ 0 //)«A//(3g m ) . 

The transconductance g m is given by 

(N5.19-9) g m = di d /de gs 

where i d is drain current and e gs is the gate potential relative to the source. The value of / 0 is 
usually on the order of 25 kHz but may be 10 or 100 times lower if source impedance is very 
high (several megohms). 

(8) For a junction field-effect transistor (JFET), there may also be current noise in the gate lead. 
The combined effect of current noise in all leads may be represented by an equivalent random 
noise current bi g in the gate lead given by 

(N5. 19-10) (bi g ) 2 = [(t v 2 f 2 C 2 /g m ) + 2q e i g ]*Af 

where C is the capacitance between gate and drain when the latter is connected to the source, 
q e is the electron’s charge, and g m is given by Eq. (N5.19-9). 

(9) For an insulated-gate field effect transistor (IGFET), the value of f 0 in Eq. (N5.19-8) is usually 
on the order of 1.5 x 10 8 g m ft*Hz; at low frequencies, the corresponding flicker noise is usually 
unacceptably large. 

A5.21 Low-level radiation detectors used in astrophysical observations are often cooled to very low 
temperatures. The cooling often extends to the first amplifier stage in the chain of signal processing. 

JV5.23 As an example of the range of considerations that may affect instrument selection, consider a problem 
of controlling the mass flow rate m of fluid in a pipe; the control of m, part of an industrial process, is 
to be made without recourse to auxiliary measurements and computations. At the simplest extreme, if the 
composition and density of the fluid will remain substantially constant, if the piping system is fixed, and 
if velocity profiles, turbulence, and swirl will remain almost invariant, the control of the pressure differential 
Ap between two appropriate locations in the piping system may be sufficient to permit maintenance of a 
desired flow rate, once the correlation has been established between Ap and either m or a final process 
variable that is uniquely correlated with m. At an intermediate level of complexity, if fluid density is always 
known, a volume-flow meter may be used that may prove to be less expensive, or easier to install, to maintain, 
or to service, than a mass-flow meter. At a higher level of complexity, the density and Prandtl number 
of the fluid and the Reynolds number of the flow may be expected to vary widely, the velocity profile may 
be changeable because of the unavoidable presence of valves or pumps, or the fluid may sometimes exist 
in two phases; thereby, only a few designs of mass-flow meter may prove suitable. 
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CHAPTER 6. INTRODUCTION TO DYNAMIC PERFORMANCE. 
EXAMPLES OF SOME SIMPLE SYSTEMS 


6.0 Introductory remarks. When the physical variable being measured by an instrument varies with time, 
one may desire either 

(i) a knowledge of the mean value of the variable over a given time interval, as deduced from the 
instrument indication, or 

(ii) a knowledge of the instantaneous value of the variable, as deduced from the simultaneous instrument 
indication. 

The first case has been considered in Chapter 5. [Note N6.0] The second case requires more precise 
definitions of instantaneity and of simultaneity. Practically, these concepts imply averaging over a time interval 
At that is very short compared to the total time of observation. The choice of At is important because it 
affects the signal-to-noise ratio (Sec. 5.20) that can be achieved. 

The second case will be illustrated in this chapter by some simple, basic systems; their behavior will 
then be treated in Chapter 7. The understanding thereby provided often serves to provide an understanding 
of the behavior of more complex variants or combinations of these systems, with little additional mathematics. 

Simplicity of treatment of the systems is achieved if results are expressed in terms of the time t and of 
the dimensionless quantity * ‘fraction of reference indication,” which will be denoted by y; it will then be 
found that only a few mathematical equations suffice to describe the dynamic performance of a variety of 
instruments, systems, or installations. The examples in this chapter illustrate this fact. The examples also 
provide an understanding of the instruments, of the factors that influence their dynamic behavior, and of 
the methods of analysis that can be applied to other situations. 

In what follows, the first or second derivative of any variable with respect to time will be denoted by 
one or two dots, respectively, over that variable. Thus, 

(6.0-1) y m dyldt; y s d 2 y/dt 2 

Generally, the equation describing the dynamic behavior of an instrument or system includes three terms 
or components. The first serves to describe the static behavior and the energy associated with the deviation 
of y from its equilibrium or static condition (the potential energy in mechanical systems). The second is 
indicative of the dissipation of energy and is usually associated with y. The third is usually associated with 
y and, in mechanical systems, with the conservative kinetic energy. 1 


l In electrical systems, the two nondissipative forms of energy are those associated with the energies stored in pure capacitances 

and in pure inductances. 
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6. 1 Linear spring-mass system. Seismometer , velocimeter, accelerometer , dynamometer , or pressure 
gauge. 

In the system shown in Fig. 6.1, the elements are assumed to be 

1. a linear spring having a spring constant k (force/distance), negligible mass, and negligible elastic 
errors; 

2. a mass m; 

3. an oil film or equivalent device having negligible mass and producing a viscous force b per unit 
velocity in the .x-direction; 

4. an external force F that acts on mass m ; 
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Figure 6.1— Linear spring-mass sysiem. 


5. an instrument base with scale that measures the horizontal deflection of mass m under force F. 
Figure 6. la shows the system at rest, with F = 0, and the scale adjusted so that pointer deflection x = 0. 
Figure 6.1b shows the system when x > 0 under the influence of the external force F = F(t). We apply 
Newton’s Second Law 

(6.1- la) mx = 12 forces. 


Then 


force exerted by oil film = -bx 
force exerted by spring = —kx 
external force = F 
so that 

(6.1-lb) mx = F -bx-kx . 

Let x m be the steady deflection under a steady force F m (conveniently, these may be taken as full-scale 
values) 

so that 

(6.1-lc) F m = kx m ; F= (F/F m )kx m 

and Eq. (6.1-lb) may be rewritten as 

(6. 1— Id) (m/k)x + (b/k)x + x = (F/F m )x m . 

Let 


(6.1 -2a) x/x m =y; 


b/k = r; 


m/k = 1/(4t 2 /J) . 
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Then Eq. (6.1 -Id) becomes 

(6.1 -2b) >’/(4 tt 2 /;) + ry + y = F/F m . 

This system serves as a model of several instruments: 

A. When the instrument base oscillates in the jc-direction at a frequency f>>f„, the model 
represents a linear seismometer. The magnitude of jc is substantially the same as the magnitude of base 
displacement; force F is substantially zero. [Note N6.1] 

B. When the instrument base oscillates in the jc-direction at a frequency f ~ f n > the model represents 
a linear velocimeter. The magnitude of the displacement x is approximately proportional to the velocity 
of the base. 

C. When the instrument base oscillates in the ^-direction at a frequency /<</„, the model represents 
a linear accelerometer. A static calibration if = 0) is possible, using the acceleration corresponding to the 
Earth’s gravity, by turning the assembly ±90° from the horizontal position shown in Fig. 6.1. When the 
base is so turned, the model also represents a spring balance. 

D. A vibration meter may use either a seismometer, velocimeter, or accelerometer as a pickup unit 
and then use electrical networks to produce single or double differentiation or integration to provide 
measurements of the other two variables. 

E. If F is an external force, the model represents a dynamometer or thrustmeter. It represents a weighing 
device if it is turned 90°; m then primarily represents the mass of the object being weighed. 

F. If the mass m is in the form of a piston in a cylinder and the force F is the product of piston area 
and pressure on the piston face, the model represents a pressure gauge. 


6.2 Correction for spring mass in Sec . 6 . 1 . The expression for f n in Eq. (6.1-2a) was based on the 
assumption that the spring had negligible mass. A more realistic expression would include the effect of 
the spring’s mass m s by adding some fraction of m s to the quantity m in Eq. (6.1 -2a). If m s is uniformly 
distributed along the spring’s length, as in a helical spring or a long bellows, then the fraction is 1/3. If 
the spring is a cantilever of constant cross section, loaded at the free end, the fraction is 33/140. If the 
spring is a circular diaphragm, and the supported mass occupies a small central area, the fraction is on 
the order of 1/4 if the outer edge is simply supported and is on the order of 1/6 if the outer edge is clamped. 
[Notes N6.2.1 and N6.2.2] 

6.3 Rotational spring-mass system. Angular vibrometer, torquemeter, or galvanometer. The elements 
treated in Sec. 6.1 may be replaced by 

1 . a torsion spring having a spring constant K (torque per unit angle of twist) and with negligible 
mass and negligible elastic errors, 

2. a mass with polar moment of inertia /, 

3. an oil film or equivalent device (like an eddy-current drag disc) having negligible mass and producing 
a resisting torque B per unit angular velocity, 

4. an external torque T e acting around the axis of the torsion spring and the center of rotation of the 
mass. 

These are illustrated in Fig. 6.3. 

The equation corresponding to Eq. (6.1-lb), that applies to the rotational system, is 


(6.3-1) Iip = T e - B<p - Kip 

which can be put in the form 

(6.3-2a) y/(4T 2 /n) + ry + y = T e /T tm 
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Figure 6.3— Rotational spring-mass system. 


by the substitutions 

(6.3-2b) <?/*„ = y; B/K = r; I/K = l/(4*- 2 /*) 

where <p„ is the steady angular deflection under a steady torque T em . The quantity T em may often 
conveniently be taken as the full-scale range of the device. 

As in Sec. 6.2, if the torsion spring has uniformly distributed polar moment of inertia I s , addition of 
I s /3 to / in Eq. (6.3-2b) would yield a more nearly correct value of f„. 

As in Sec. 6.1, if the base undergoes a periodic oscillation at frequency /, the system is a model of a 
vibration meter whose mechanical deflection would be proportional to angular amplitude, angular velocity, 
or angular acceleration, depending on whether /> > /„, /*/„, or f <<f„> respectively. 

If the mass is subjected to a couple (e.g., if the mass is a motor armature or rotary-dynamometer armature), 
so that the twist of the spring is proportional to the magnitude of the couple, we have a model of a torquemeter. 

The system is also a model of a d’Arsonval galvanometer , where B is the sum B\ + B 2 of two factors, 
each of which arises from the facts that the motion of a conductor in a magnetic field generates voltages 
proportional to the velocity of relative motion and that these voltages generate currents whose reactions 
with the magnetic field produce forces that tend to oppose (damp) the motion. The factor B x is due to eddy 
currents generated in the moving-coil support and is not under the control of the user. The factor B 2 is 
due to the current in the coil, and is inversely proportional to the total resistance of the circuit of which 
the coil is a part; thus, B 2 is under the control of the user. 

In the d’Arsonval galvanometer, a current i produces a torque proportional to i. If this torque is opposed 
by a mechanical moment FR produced by a force F acting at the end of an arm of radial length R, the 
arrangement is a model of an electrobalance , in which current i is adjusted until = 0. Then i is proportional 
to F. 

6.4 U-tube manometer . In the manometer shown in Fig. 6.4, liquid of density p and viscosity t/ in a 
tube of inside radius r is subjected to a pressure difference P. The total length of the liquid column, from 
meniscus to meniscus, is /; the curvature of the U-tube is assumed sufficiently small so that the effect of 
accelerations caused by curvilinear motion can be neglected; and Poiseuille’s law is assumed to be 
applicable. 2 We apply Newton’s Second Law to the liquid column when P is changing and one meniscus 
is deflected a distance x from the equilibrium position corresponding to P = 0. 

mass of liquid column = par 2 / 

force exerted by gravity = - 2pgxr 2 x 


2 The assumption is valid only when l/r is very large and when the flow is always laminar. 
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I 


force exerted by viscous drag = -8ttt}Ix 
force exerted by external pressure = irr 2 P 
Therefore, 

(6.4- la) pirr 2 lx = - lpgirr 2 x - 87r rjlx -1- tt r 2 P . 

If x m is the steady deflection corresponding to a steady pressure difference P m , so that 

(6.4- lb) 2 pgx m = P m , 

then Eq. (6.4- la) may be written in the form 

(6.4-lc) (l/g) x + [8^//(gr 2 )]i + 2x = 2x m (P/P m ) , 

where f is the kinematic viscosity of the liquid, or 

(6.4-2a) y/(47r 2 / 2 ) + ry + y = P/P m 

where 

(6.4-2b) y=x/x m ; r = 4 vl/(gr 2 ); l/(4x 2 /^) = //(2^) 



Figure 6.4.— U-tube Figure 6.5 — Well-type manometer. 


manometer. 

6.5 Well-type manometer . In the manometer shown in Fig. 6.5, the ratio A { /A 2 between tube area and 
well area is made very small, so that 

(i) deflection x for a given pressure differential is almost twice what it would be in Fig. 6.4; 

(ii) the requirement in Fig. 6.4 that both legs of the U have exactly the same area or else that the 
displacements of both menisci be measured, is avoided. 

Equations (6.4-lc) and (6.4-2b) still apply provided that / is replaced by 

2(/ + jc)/(1 -F A } /A 2 ) ; 

but the equation may be treated as linear (i.e., t and f n may be treated as constants) only when oscillations 
Ax about the average position are very much smaller than (/ + jc). 

The static calibration analogous to Eq. (6. 4- lb) is 

(6.5-1) pgx m { 1 4- A x /A 2 ) = P m [Footnote 3] 

3 Thc scales of industrial well -type manometers are appropriately marked to correct for the area ratio and so directly to indicate 
the correct pressure. 
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6.6 Basic pressure transmission system. In the system shown in Fig. 6.6, the components are assumed to be 

1 . a chamber containing a volume V of a compressible gas with specific-heat ratio 7, and initial 
pressure p 0 ; 

2. a long capillary tube, of cross-sectional area A , length /, filled with an incompressible fluid of 
density p, viscosity 77, and kinematic viscosity v; 

3 . a pressure P = P(t) applied to the open end of the tube. 



^x — 0 -X — 0 

(a) P = po- (b) P> po- 

Figure 6.6— Pressure transmission system. 


If we assume an adiabatic process within the chamber and Poiseuille’s law within the tube, and if the 
position of the end of the fluid “piston” in the tube is assigned the coordinate x = 0 when external pressure 
is steady and equal to p 0 , the system becomes similar to the spring-mass system of Sec. 6 . 1 , wherein the 
fluid in the tube becomes the mass m and the compressible gas in the chamber becomes a spring such that 
a piston displacement jc (that produces a volume change Ax) also produces a pressure change -{yp$/V)Ax. 
We thus have, if an external pressure P produces a piston displacement jc, 

mass of piston = Alp 

viscous drag force on piston = — %ttt)Ix 

pressure in chamber = p 0 + (yp 0 /V)Ax 

external pressure = P = P(t) 

Application of Newton's Second Law leads to 

(6.6- la) Alpx = [P - p 0 ~ (ypo/V)Ax]A - 8 tt jlx 

or 

(6.6-lb) Ipx + %ttjIx/A + (7 Pq/V)Ax = P - p 0 . 

These equations may be converted into a description of the pressure p in the chamber by means of the adiabatic 
relation 

(6.6- 1 c) p = po + Axyp 0 / V 
leading to 

(6.6-ld) [plV/(Ayp 0 )]p + [StjjIV/ (A 2 yp 0 )]p + p-P . 

If both sides of the equation are divided by a reference pressure p m , there results 
( 6 . 6 - 2 a) y/ ( 4 ir 2 /^) + ry + y = P/p m 
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where 

(6.6-2b) 1/(4 tt 2 /«) = plV/(Ayp 0 ); t = SttijIV/ (A 2 yp Q ); y = p/p m . 

Equations (6.6-1) and (6.6-2) apply also when the tube contains the same gas as the chamber, if tube 
volume is negligible compared to chamber volume. However, it is then usually more convenient and more 
meaningful to express the quantities f n and r in terms of the speed of sound a and the kinematic viscosity v: 

(6.6-2c) l/(4x 2 /^) = lV/(Aa 2 ); r = iicvlV/ (A 2 a 2 ) 

These equations apply even to oval tubes. 

As in Sec. 6.5, the quantities/, and r may be treated as constants only when the fluctuations in P are 
small compared to p 0 . [Note N6.6] 

6.7 Modifications of the basic pressure transmission system. A number of elaborations of the basic system 
treated in Sec. 6.6 lead to several cases of practical interest. Combinations of these cases can also be treated, 
often by superposition. In the cases treated below, it will be assumed that the tube contains the same 
compressible gas as that in the chamber. The cross-sectional area of the tube will be denoted by A. 

A. If the volume of the tube is small, but not negligible compared to the volume of the chamber, 
replace V in Eq. (6.6-2) with V + All 2. 

B. If the volume of the tube is comparable to or larger than the volume of the chamber, replace V 
in Eq. (6.6-2) with V + AAl/it 2 . This substitution extends even to the case when V — 0, in which case 
we have a closed-end organ pipe. 

C. If / is on the order of magnitude of r, the effective radius (twice the hydraulic radius) of the tube, 
and if Al < < V, replace / with (/ + 7rr/2) in the expression for f n In Eq. (6.6-2) and assume r = 0. [Note 
N6.7] The system is called a bottle resonator or a Helmholtz resonator with short neck . 

D. If / < < r, replace / with 7rr/2 in the expression for /„ in Eq. (6.6-2) and assume t = 0. [Note 
N6.7] This system is called a Helmholtz resonator. 

E. If the chamber wall is flexible, so that, at average pressure p 0 , a pressure increase Ap produces 
a chamber-volume increase a 9 Ap, then replace V with (V + aypo) in Eq. (6.6-2). This model serves well 
when a pressure gauge is of the bellows or aneroid type. 

6.8 Series electrical circuits. In the circuit of Fig. 6.8, containing inductance L, capacitance C, and 
resistance R in series with each other and with an externally applied voltage E , Kirchoff s law requires 
that the algebraic sum of all voltage drops around the loop be zero. Then, if charge q is circulating through 
the system to produce a current i, 

voltage across L = e L = L(di/dt) = Lq 

voltage across R = e R = Ri = Rq 

voltage across C — e c = qfC 

so that 

(6.8-la) Lq + Rq 4- q/C = E 
or, since q = Ce c , 

(6.8-lb) LC e c + RC e c + e c = E . 
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If we divide through by some constant reference voltage e m , we may write 
(6.8-2a) y/(4ir 2 f 2 „) + ry + y = E/e m 

where 

(6.8-2b) 1/(47 r 2 f 2 n ) = LC; r = RC; y = e c /e m . 

R-C Circuit 

If inductance L is not present in the circuit, then Eqs. (6.8-2a) and (6.8-2b) become 
(6.8-3a) ry + y = E/e m 
where 

(6.8-3b) r = RC; y = e c !e m . 

R-L Circuit 

If capacitance C is not present in the circuit, then Eq. (6.8-la) can be written as 
(6.8-4) (L/R)e R + e R = E 

and division of each term by some reference voltage e m yields 

(6.8-5a) rv + y = E/e m 

where 

(6.8-5b) t = L/R; y = e R /e m . 



Figure 6.8 — Series Figure 6.9 — Parallel 

electrical circuit. electrical circuit. 


6.9 Parallel electrical circuits. In the circuit of Fig. 6.9, wherein a current / passes through the parallel 
combination of inductance L, capacitance C, and resistance R. KirchofFs law requires that the sum of all 
currents at point P ] shall be zero. If the voltage between points P t and P 2 is E. then 

(6.9-la) E = L(di L /dt) = Ri R = ( 1 /C)q c = (1/C) |i c dt 

and KirchofFs law 
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(6.9-lb) j c + i R + i L = I 
can be written as 

(6.9-lc) LC(d 2 i L /dt 2 ) + (L/R) (di L /dt) + i L = 1 . 



If we divide through by i m , this may be written as 


(6.9-2a) y/(4ir 2 /;) + tv + >’ = I/i m 
where 

(6-9-2b) 1/(4 v 2 f 2 n) = LC; r = L/R; y = i L /i m . 

R-L Circuit 

If capacitance C is not present in the circuit, then Eqs. (6.9-2a) and (6.9-2b) can be written as 
(6.9-3a) ry + y = l/i m 

where 

(6.9-3b) r = L/R; y = i L /i m . 

R-C Circuit 

If inductance L is not present in the circuit, then Eq. (6.9- lb) can be written as 
(6.9-4) RC(di R /dt) + i* = / 

and division of each term by some reference current i m leads to 

(6.9-5a) ry + y = I/i m 

where 

(6.9-5b) r = RC; y = i R /i m . 


6.10 Mass moving through a viscous medium. If a force F causes a mass m to move through a viscous 
medium that exerts a resisting force b\ proportional to the velocity v of motion, application of Newton’s 
law yields 

(6.10-la) mv = F — bv 

Under a steady force F m , the mass would ultimately acquire a constant velocity v m given by 
(6.10-lb) F m = b\ m . 

Consequently, Eq. (6. 10- la) may be written as 

(6. 10-2a) ry + y = F/F m 

where 

(6.10-2b) r = m/b ; y = v/v m . 

These relations are applicable to the phenomenon of sedimentation under the action of gravity and to 
the falling-ball viscometer. 
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6.11 Filling of a gas container . In Fig. 6.11, a chamber of volume V, initially at pressure p 0> is filled 
through a restriction connecting the chamber to a source of higher pressure P. The following conditions 
are assumed to exist: 

1. The fluid is a perfect gas. 

2. Gas motion is so slow that inertia effects can be neglected. 

3. The exchange process is isothermal, at temperature 0. 

4. The restriction through which the filling takes place is a linear one, in which mass flow rate m 
through the restriction is proportional to pressure difference P — p, where P is upstream pressure 
and p is the pressure in the chamber: 

(6.11-1) m = 0(P — p) 

where C has the dimensions of (mass flow rate)/ (pressure difference). [Note N6.11.1] 



Figure 6. 1 1.— Filling of a gas container. 


Since the rate of accumulation of mass in the chamber is 
(6.11-2) m = = VpM/(R<fi) , 

where M is relative molar mass and Rq is the universal gas constant, the law of conservation of mass 
requires that 

(6.11-3a) VpM/ (R qO) = C 9 (P — p); C « dm/dp 
so that 

(6. 11 -3b) [VM/(CRoP)lp+p . 

Dividing through by some reference pressure p m leads to 
(6. 11 -4a) ry + y = P/p m 
where 

(6. 1 1 —4b) r=VM/(CRd9); y = plp m . 

These results are applicable to the calibration of standard leaks and flow impedances. [Note N6.11.2] 
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Example 6.11 a. If the restriction is a capillary tube of cross-sectional area A, and if/>o > 1 mbar, 

C = A 2 / (Snlv) 

where the kinematic viscosity v must be chosen as the value at a particular location. [Note N6. 1 1 .3] 
Then 

t = 8ttpIVM/(A 2 R 0 6) . 

Example 6.11 b. If the restriction is a knife-edged orifice of area A. and if Po < 1 /xbar, 

C — 2AI(ttc a ) 

where c A is the arithmetic mean speed of the molecules and is given by 

c A = 8Ro0/(irM) . 

Then 


r = 4 V/(Ac a ) . 

(c A is 468.0 m/s if M = 29.0 and $ = 300 K.) 

6.12 Thermometer bulb . Assume that a thermometer bulb of mass m , specific thermal capacity c (energy 
to produce unit temperature rise in unit mass), very high thermal conductivity, and exposed surface area 
A is at temperature 0 and receives heat through its exposed surface by conduction along a column of matter 
of cross-sectional area A, length /, and thermal conductivity k, whose far end is maintained at temperature 0 
(Fig. 6.12a). At the interface between the column and the bulb, the rate of heat conduction is equal to the 
rate at which heat is stored in the bulb. 

Rate of heat conduction through the column = kA(B — 6) /l 
Rate of storage of heat in bulb = mc*dQ/dt 
Consequently, 

(6.12-1) mcB = kA(G - 0)/l . 

If each term is divided by some reference temperature B m , the relation may be expressed as 

(6. 12-2a) ry + y = 0/0 m 

where 

(6. 12-2b) r = mcl/ ( kA ) ; y = 6/6 m . 

If the means of transferring heat to the bulb is by gas convection (Fig. 6.12b) and the convective heat 
transfer coefficient is h (power per unit area per unit temperature difference), then the equality of heat transfer 
rate and heat storage rate is expressed by 

(6.12-3) mcO = M(0 - 0) . 
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(a) Conductive heat transfer. 

(b) Convective heat transfer, 
id Radiative heat transfer. 

Figure 6. 12.— Thermometer bulb. 


Division of each term by some reference temperature 0 m i*ads to 

(6.12-4a) ry + y = 0/6 m 

where 

(6.12-4b) t = mc/(hA); y = 0/0 m . 

If the surface of the bulb has an emittance t and if the bulb is surrounded by an enclosure at temperature 
0 (Fig. 6.12c), the rate of energy exchange by radiation is such that 

(6.12-5) mcd = oeA(Q 4 — 0 4 ) 

where a is the Stefan-Boltzmann constant. If (© — B ) /0 < < 1, an approximation is 
(6.12-6) mc6 = 4oeAQ 3 (B - 6) . 

Division of each term by some reference temperature 8 m leads to 

(6.12-7a) ry + y = B/8 m 

where 

(6.12-7b) t = mc/(4oeAB 3 ); y = 6/6 m . 


144 




§6. 13-Notes 


6.13 Summary . All of the examples of simple dynamical systems in this chapter have led to the expressions 
of the form 

( 6 . 13 - 1 ) y/(4* 2 f 2 n ) + ry + y= Y 

or of the form 
( 6 . 13 - 2 ) ry + y = Y 

where y is a dimensionless quantity representing instrument indication and Y is a dimensionless quantity 
representing the physical variable being measured. The parameter f n will be termed the natural 
frequency”; the parameter t will be termed the ‘‘time constant”. These parameters are significant only 
when Y is varying with time; if it is not, then we have the static conditions considered in Chapter 5 . 
The values of/* and r for some of the particular systems treated here are summarized in Table 6 . 13 . 


TABLE 6. 13. -PARAMETERS OF SOME DYNAMICAL SYSTEMS 


System 

Variable 

r 

l/(4ir J /^) 

Linear spring-mass 

X 

b/k 

m/k 

Torsional spring-mass 


B/K 

l/K 

Manometer 

X 

4t '//(gr 2 ) 

l/(2g) 

Pressure transmission 

P 

8* vlVtKAW) 

IV/ (Aa 2 ) 

Series L, R, C 

e C 

RC 

LC 

Series R, C 

€ c 

RC 


Series L, R 


L/R 


Parallel L, R, C 

k 

L/R 

LC 

Parallel L, R 

[l 

L/R 


Parallel R, C 

ir 

RC 


Mass in viscous medium 

V 

m/b 


Filling a gas container, continuum 

p 

8irWt7M 2 *o0/Ari 


Filling a gas container, free-molccule 

p 

4 V/(Ac a ) 


Thermometer bulb, conduction 

e 

mclf (kA) 


Thermometer bulb, convection 

e 

me/ {hA) 


Thermometer bulb, radiation 

e 

me/ (4aH9 3 ) 



NOTES FOR CHAPTER 6 

N6.0. Considerations of linearity are important. Thus, a correct arithmetic mean will be obtained if the 
instrument indication varies linearly with the variable being measured; if it does not, then the desired accuracy 
can be achieved only if the variations of the variable are of sufficiently small magnitude. 

N6.1. “Seismographs” used to record earth motions that occur in earthquakes may actually be velocimeters 
or accelerometers because the amount of earth displacement may be too large to be recordable by any practical 
seismometer. 

N6.2.1. More precisely, if a = outer edge diameter and b = central area diameter, the fraction is given 
in the following table 


alb - 

Supported edge 
Clamped edge 


5 10 20 
0.31 0.27 0.25 

0.20 0.16 0.14 
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N6.2.2. When a flush-diaphragm-type pressure gauge is immersed in a liquid, the mass of the liquid next 
to the diaphragm becomes an additional part of the distributed mass m s . An approximate estimate of the 
effect of this associated liquid mass can be made by assuming that a hemisphere of liquid, erected on the 
diaphragm, moves with the diaphragm. 

N6.6. It is sometimes helpful to replace Eq. (6.6-1) with an equation for the volumetric displacement 


u = Ax 


or an equation for the mass displacement 


M = pAx . 


The respective equations then are 

(N6.6-1) (lp/A)u + (Sx V l/A 2 )u + (yp 0 /V)u = P - p 0 
and 

(N6.6-2) (t/A)M + (8m>l/A 2 )M + (a 2 /V)M = P - p 0 . 

Equation (6.6-2) remains unchanged. 

N6.7. The effect of energy dissipation because of fluid viscosity is usually negligible. If the resonator opens 
into a medium of substantially infinite extent, any acoustic oscillations that exist after removal of the initiating 
source of energy will die out principally because of the radiation of energy into this infinite medium. 
The effective value of 7 is given by 


t * A/ (Aval) 


for a resonator with short neck, and by 


r 2 ^ A/(4v 3 a 2 ) 

for a resonator with neck of negligible length. 

N6.11.1. The value of C depends on the geometric form of the restriction and on the intrinsic and extrinsic 
properties of the gas. 

Typical intrinsic properties are viscosity, relative molar mass (molecular weight), and Prandtl number. 
Typical extrinsic properties are density, temperature, pressure, speed of sound, and Reynolds number. 

N6.11.2. Thereby, it is not necessary to measure mass flow rate in order to determine the flow resistance C. 
It is necessary only that V and t be accurate, that P be constant, and that the pressure gauge be linear. In 
some practical applications, it may be necessary to correct for any rate of pressure rise in Kdue to outgassing 
from the inner surface of the chamber. 

N6.11.3. The location preferred is that at which v is likely to change least during the course of the filling 
operation. Usually, this location is the entrance end of the capillary. Other locations are (a) the exit, or 
(b) a point where the pressure is the mean of the entrance and exit pressures. 

Appreciable changes in kinematic viscosity during the course of the filling operation make Eq. (6. 1 1-4) 
nonlinear; such nonlinearity is discussed in Sec. 10.3. 
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7.0 Introduction and terminology. A study of the behavior of systems described by Eqs. (6.13-1) and 
(6.13-2) is useful because 

(a) the behavior is completely characterized by no more than two independent parameters and hence 
can be easily understood and remembered; 

(b) more complex systems can often be represented as combinations of the systems described by 
Eqs. (6. 13-1) and (6.13-2) so that an intuitive understanding of the behavior of the more complex 
system can be derived from an understanding of the behavior of the simpler component systems; 

(c) an understanding of dynamic behavior helps the attainment of several possible goals: 

(i) adequately accurate determination of the average value of a quantity that is changing with time, 

(ii) adequately accurate determination of the instantaneous value of such a quantity, 

(iii) adequate suppression of random or systematic fluctuations (usually categorized as “noise” 
or “hum”) that tend to prevent such accurate instantaneous measurements. 

In practical situations where dynamic measurements are to be made, the balance between “signal” and 
“noise” is always necessary, and requires a human judgment that also considers convenience, cost, and 
reliability. 

The following symbols will be used frequently in this chapter: 

/ frequency of Y(t) 

f c comer frequency 

f n natural frequency 

f r resonant frequency 
I impulse 

/ time 

T period of Y(t) 

T n natural period 

T r resonant period 

y dimensionless variable proportional to the instrument indication (instrument response) 

Y dimensionless variable proportional to the physical quantity being measured. Y(t) is also 
called the forcing function. 

a V|l-f 2 | 

€ base of Napierian logarithms 

A radius of curvature 
r time constant 

f fraction of critical damping 

o) angular frequency of Y(t) 

u n natural angular frequency 
u r resonant angular frequency 

Other symbols used occasionally will be defined as they are introduced. As in Chapter 6, one or two dots 
over a variable denotes the first or second derivative with respect to time. 
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7.1 Basic equations . The basic first-order system will be defined as one obeying the equation 
(7.1-1) ry + y = T = Y(t) 
where r is a constant. 

The basic second-order system will be defined as one obeying the equation 
(7. l-2a) y/w* + ry + y = Y m Y(t) . 

This equation can be written in several other forms, three of which are shown here 


(7.1 -2b) 

y/ (4ir 2 f„) + ry + y = Y 

(7. 1— 2c) 

y/oil + (2 £/«„)>’ + y = Y 

(7.1 -2d) 

>"7-2/ (4x 2 ) + (f TJtW + v = Y 


The various constants are related by 
(7.1-3) o>„ = 2*/„ = 2*/T n ; r = 2{/u H = f T n /* . 

The dynamic behavior problem is treated in two steps: 

1 . Given T(r) , one finds y(t) . The difference (y-T) represents the instantaneous error of measurement. 

2. One then determines what values of the parameters in (7. 1-3) will render the error acceptably small. 

7.2 Parameters affecting response. The response of a first-order system to any Y(t) is determined entirely 
by the value of r. The response of a second-order system to any Y{t) is determined entirely by two parameters: 
one of these is chosen from the group (/„, T n , a> n ); the other is either r or f. When f £ 1, it is usually 
more convenient to use f in mathematical descriptions of the response and to express time in terms of T n . 
When f > 1, it is usually more convenient to use r; in fact, when T > > L it is often adequate to treat 
the system as a first-order system described by Eq. (7.1-1). 

7.3 Influence of prior values of y and its derivatives. Assume that Y(t) is known for t > f 0 . Let 
yo * y(*o)> jo * j(*o)> yo * j(*o)- If the system is a first-order one, y(t) for t > r 0 will depend only on 
y 0 . If the system is a second-order one, y(r) for r ^ r 0 depend only on y 0 and y 0 - If the system is an 
n^-order one, y(t) for t > / 0 will depend on y 0 and on the first (n-1) derivatives of y at time to* 

An alternative phrasing is to say that y (f) for a first-order system is capable of executing abrupt changes 
in direction (i.e., y must be continuous, but y need not be continuous); in a second-order system, abrupt 



Figure 7.3.— Response to a square wave of two 
systems with the same time constant, r. 

(a) first-order system: (b) critically-damped 
second-order system; (c) forcing function with 
period 2 r. 
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changes in direction cannot occur, although abrupt changes in the second derivative may occur (i.e., v and 
v must be continuous, but y need not be continuous). This behavior is illustrated in Fig. 7.3, which shows 
the response v of first-order and second-order systems to abrupt changes in Y. Sharp corners in y ( / ) are 
possible only for a first-order system; they are not possible in systems of higher order. In an /? th -order 
system, abrupt changes (sharp corners) are not possible in the first (n-1) derivatives. 

7.4 Higher-order systems. If the dynamic behavior of a physical system is described by a linear differential 
equation of the « th order, that description may be rewritten as a combination of first-order and second- 
order equations representing corresponding first-order and second-order subsystems. In one such combination, 
the subsystems are in tandem, so that the indication (output) of one subsystem becomes the input of the 
next subsystem; the output of the last subsystem is the desired response y of the entire system to the initial 
input Y. Such a combination may be termed a 4 ‘serial combination". 

In another combination, appropriate modifications of the input Y are delivered to each subsystem and 
the outputs of all subsystems are summed to yield the indication y. Such a combination may be termed 
a “parallel combination”. [Note N7.4] 

7.5 Transient and steady-state responses . For any physical system, the response y(r) to an applied Y(r) 
is often conveniently treated as composed of two parts: a transient response whose magnitude and character 
depend on the conditions (termed the “initial conditions”) existing at time / = t 0 when Y was first applied; 
and a steady-state response whose magnitude and character are not influenced by the initial conditions and 
depend almost entirely on the current behavior of Y(t), The terms “transient” and “steady state” are used 
because, in most cases, the transient response is of greatest magnitude during the time immediately after 
t 0 an d ultimately dies out (i.e., becomes negligibly small after a sufficiently long time has elapsed), whereas 
the steady state remains after the transient has died out and persists as long as T persists. In a strict mathematical 
sense, the steady-state response is an asymptotic condition approached as t — oo. Practically, it often becomes 
indistinguishable from the asymptotic value after a time equal to a few multiples of t. 

The applied quantity Y, which is also termed the “forcing function”, represents the true value of a quantity 
to be measured. The response y represents the indicated value of that quantity. In any relation of the form 

(7.5-la) y = Y- A , 

A represents the correction to be added algebraically to the indication y in order to find the true value Y. 

An elaboration of Eq. (7.5- la) is 

(7.5- lb) y= Y- A SS -A T 

where A ss and A T represent the steady-state and the transient corrections, respectively. Their distinctive 
features are summarized here. 

A ss : 

(1) Mathematically, it is an asymptote approached as / — oo. 

(2) Practically, it may be approached to within acceptably small limits of deviation in a time determined 
by the values of the parameters listed in Sec. 7.1. 

(3) It does not depend on the conditions existing at the initiation of the measurement. 

(4) In most practical cases, it is of sufficiently simple form so that the application of corrections 
is convenient. 

A t : 

(1) Mathematically, A T — 0 as t — oo. 

(2) Practically, it reaches any specified small magnitude in a time determined by the values of the 
parameters listed in Sec. 7.1. 

(3) It depends on the conditions existing at the initiation of the measurement. 

(4) When it is of sufficiently simple form, it may be convenient to apply it as a correction to Y in 
order to deduce y\ at other times, it may be more convenient to wait until it becomes negligible 
in comparison with y + A ss before the deduction of Y is attempted. 
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If the steady-state response is known for each of several forcing functions, like those treated in the sections 
that follow, the steady-state response to the sum of the individual forcing functions is merely the sum of 
the individual steady-state responses. 

The transient responses may not be treated in this additive manner. 



7.6 Forms of the forcing function. In the remainder of this chapter, both transient and steady-state responses 
will be indicated for several forms of the forcing function Y . These are illustrated in Fig. 7.6. For ease 
and clarity of presentation, the initial time t 0 will always be taken as / = 0. 

(a) Step. A sudden change in Y from an initial constant value Y 0 to another constant value Y\. 

(b) Exponential. An approach of Y from an initial constant value Y 0 to an asymptotic value T, in 
an exponential manner. 

(c) Ramp. A linear change of Y with time, starting from an initial constant value Y 0 . 

(d) Quadratic parabola. A parabolic change of Y with time, starting from an initial constant value 

of T 0 . 

(e) Impulse. A momentary step change in Y, from an initial value Y 0 to a final value Y \ , that is of 
such short duration At that At 0, Y x ~ oo, but (Y x - Y 0 )^At = / is finite. The product I is 
termed the “impulse”. 

(0 Sinusoid. A sinusoidal variation in Y with amplitude Y x and average value Y 0 . This form is 
particularly useful because any periodic, continuous physical phenomenon can be represented by 
a summation of sinusoidal signals of appropriate phase and amplitude. 



Figure 7.7 — Response of a first-order system 
to a step change. 


7.7 Response of a first-order system to a step change. (Fig. 7.7) If 

(7.7-1) Y=Y l for r > 0 

and Y = y - y 0 at t - 0 

then 

(7.7-2) y = Y x - (T, - Y 0 U~ t/T 
is the complete solution. 
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The asymptote is Y { . 

The initial slope y(0) is (Y { — Y 0 )/t. 

If the tangent dy/dt is drawn at t = 0, it will intersect the asymptote Y } at / = r. 

In time r, ( v — T 0 ) will 0 ~ 1/e ) ( F, — y 0 ); in other words, 63.2 percent of the step will have been 
completed. In time 27, 3 7 , 47, 57, approximately 86, 95, 98, 99.3 percent, respectively, of the step 
(Ti — Y 0 ) will have been completed. [Note N7.7] 

7.8 Response of a first-order system to an exponential change . (Fig. 7.8) If 


(7.8-1) 

II 

1 

JN 

- y 0 )f ,/x for f > 0 


and 

n 

>> 

II 

at / = 0 


then 




(7.8-2a) 

>■= r. - W 

- r 0 )(Xe- ,A - Tt -,/T )/(X - r) 

if X * 7 

(7.8-2b) 

> = r, - o', 

- y 0 ) ( 1 +t/T)e~' /r 

II 


The response y starts out at zero slope and passes through a point of inflection at ( ) • After this point, 
y approaches Tj asymptotically in a manner resembling an exponential curve with time constant on the order 
of (7 + X). [Note N7.8] 


i ' 



Figure 7.9.— Response of a first-order 


Figure 7.8 .— Response of a first-order system to a ramp, 

system to an exponential forcing 
function. 


7.9 Response of a first-order system to a ramp. (Fig. 7.9) If 


(7.9-1) 

Y = Y 0 4- at 

for t > 0 

and 

II 

II 

at t - 0 

then 



(7.9-2) 

y = Y 0 + a(t - 

t) -F a7e ~ tlT . 


The steady state is a ramp parallel to Y but lagging behind it by the constant lag time 7. The term containing 
the exponential is the transient response; the response y starts out with zero slope at t = 0 and approaches 
the steady-state condition. 
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At times r, 2 r, 3 7, 4r, 5r, the respective lag time between y and T is 63, 86, 95, 98, 99.3 percent of 
r. (These percentages are the same as those quoted in Sec. 7.7). The differences between the ordinates 
at these same times is ar times the respective percentages. [Note N7.9] 

7.10 Response of a first-order system to a quadratic forcing function . (Fig. 7.10) If 


(7.10-1) 

Y = y 0 + bt 2 

for t > 0 

and 

II 

II 

£5. 

II 

0 


the forcing function is a quadratic parabola with vertex at (0,y 0 ). The response is given by 
(7.10-2) y = Y 0 + br 2 + b(t - t ) 2 - 2 £tV / t . 


The steady state is a quadratic parabola of the same shape as the forcing function but with the vertex 
shifted to ( t,Y 0 + br 2 ). This vertex lies on the original forcing function. The lag between the response and 
the forcing function approaches r, regardless of the initial value of y. 

The term containing the exponential represents the transient response. At times t = r, 2 r, 3r, 4 t, 5 r, 
the respective lag between y and Y is 0.497, 0.697, 0.797, 0.847, and 0.897. 


Y. with vertex 
shitted to 



Figure 7. 10.— Response of a 
first-order system to a 
quadratic forcing function. 



Figure 7. 1 1 . — Response of a first-order 
system to an impulse. 


7.11 Response of a first-order system to an impulse . (Fig. 7-1 1) An impulse is defined as a change in 
Y that is of very high amplitude and very short duration (very much less than 7), but such that the area 
under Y{t) is a finite constant I. 

A rectangular pulse of very short duration At may be described by the equations 


(7. 11-la) 

<3 

"'V 

II 

>? 

1 

X, 

for 0 < r < Ar < < 7 

(7.11-lb) 

II 

II 

at t — 0 


N- 

II 

& 

for t > At 


The response y will reach a maximum value of Y 0 4- l/r at the end of the interval Ar; thereafter, it will 
decay exponentially to an asymptote Y 0 , behaving as though it were responding to a step change from an 
initial value of Y 0 4* l/r to a final value of Y 0 , in accordance with Sec. 7.7. [Note N7.ll] 
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Figure 7. 12. 1 .-Steady-state response of a First-order 
system to a sinusoid. 


7.12 Response of a first-order system to a sinusoid. (Fig. 7.12.1) For this forcing function, only the 
steady-state response will be considered. The transient response is only of academic interest because it depends 
on the exact phase of the sinusoid at the instant that it is imposed on the system [Note N7.12.1]. If the 
forcing function Y is described by the equation 

(7.12-1) T- Y 0 = Y x sin(2ir/r) = Y x sin(o )t), 

the steady-state response will be given by 

(7.12-2a) v — Y 0 — jj sin(o)/ — <t>) 

= y y sin[a >(t - X)] 

where y\/Y { is termed the amplitude ratio 
ip is termed the phase angle 
X is termed the time lag 
These quantities are given by 

(7. 12-2b) y,/y, = (1 + co 2 r 2 )- ,/2 

(7.12-2c) ip = tan _1 (aJ7) = cos _1 (l + a rr 2 ) _1/2 

(7.12-2d) X = (1/co) tan _1 (u?r) 

The dependence of these three quantities on the value of cjt is shown in Fig. 7.12.2. 

An alternative way of graphing the steady-state response is to make a polar plot of amplitude ratio versus 
phase angle. Such a graph is termed a Nyquist diagram. For the first-order system, the diagram is shown 
in Fig. 7.12.3. It is a semicircle based on the radius vector y\/Y x = 1 at angle 0 = 0, and lies entirely 
in the first negative quadrant. 

From Eq. (7.12-2) and Figs. 7.12.2 and 7.12.3, some useful conclusions, that depend on the value of 
W7, are indicated here: 

(1) If o)7 < < 1, the amplitude ratio is reduced only slightly (often negligibly) and the time lag is 
approximately equal to r. 

(2) If cj7 = 1, the amplitude ratio is 1/V2, the phase angle is 45°, and the time lag is tt 7/4. The 
amplitude ratio and the phase angle are independent of 7. 

(3) If o)7 > > 1 , the amplitude ratio is approximately l/(o)7), the phase angle is almost 90° , and the 
time lag is approximately 1/(4 f). On the logarithmic graph of Fig. 7. 12.2a, the asymptote of the 
graph is a line of slope -1; this asymptote intersects the ordinate y\/Y x = 1 at wt = 1. The 
frequency f c at which this intersection occurs, given by 

(7.12-3) 2ir/ c 7=l, 
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Figure 7. 12.2.— Amplitude ratio y j/Y | . time lag X. and phase angle 
<t> of a first-order system s response to a sinusoid. 



90* 


T 

r 


Figure 7. 12.3 — Amplitude ratio y^/Y i versus phase angle of 
response to a sinusoid by a first-order system. 


is the comer frequency . The asymptote with slope - 1 is said to have 4 ‘a slope of 6 dB per octave’ ’ because, 
along this straight line, doubling the frequency would produce a 6-dB reduction in (y\/Y { ) 2 , which, in 
several of the physical situations treated in Chapter 6, would be proportional to power. 

In simplified descriptions of the response of a first-order system to a sinusoid, the graph of Fig. 7.12.2a 
is replaced by two straight lines: a line of slope - 1 passing through the point (1,1) for wt ^ 1, and a horizontal 
line through the same point for cor ^ 1 . 
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In an electrical circuit involving capacitance C and resistance R t the quantity 
(7.12-4) D = u)t = uRC 

may be a distinctive parameter of the circuit. In particular, if R s is the effective series resistance of an 
imperfect capacitor of capacitance C, the quantity a )R S C is the dissipation factor of the capacitor. 
[Note N7. 12.2] 

In an electrical circuit involving inductance L and resistance R , the quantity 
(7. 12-5) Q ~ oot = t oL/R 

may be a distinctive parameter of the circuit. In particular, if R s is the effective resistance of an imperfect 
inductor of inductance L, the quantity a )L/R S is the storage factor of the inductor. [Note N7.12.3] The 
dissipation factor of an inductor is the reciprocal of the storage factor. 

7.13 Response of a first-order system to any forcing function. If a forcing function Y{t) can be represented, 
over some time interval, by a sum of sinusoids 

(7.13-1) Y - EYj sin (ay - fj) 

and if only terms for which uf < < 1 have significantly large amplitude Y Jt then the record y(t) yielded 
by an instrument with time constant t can be used to approximate Y(t) by the formula 

(7.13-2) y(r)~y(r + r) . 

This situation is common in instrumentation because an instrument is usually selected, a priori , to provide 
good fidelity in measuring Y. Equation (7. 13-2) then provides a first-order correction to improve accuracy. 
In practice, the validity of the procedure is established as follows: 

(1) The record of y(r) is inspected to identify oscillation frequencies that may be apparent. 

(2) It is verified that the corresponding apparent amplitudes of these oscillations are significantly 
large only for frequencies for which o of < < 1 . 

(3) It is further verified that y (r) shows no sharp comers, representing abrupt changes in direction, 
that would imply the existence of transients of appreciable magnitude. 

The following alternative procedure is useful when no oscillation frequencies are readily identifiable, 
but there remains the question of whether Y(t) is deducible from y(r); 

Given the Cartesian graph of y (t) produced by a system of known time constant r, find the radius A min 
of the smallest circle of curvature of y(t), the radius being drawn in the direction of the time axis, so that 
A min is expressed in units of time. Let 7j 0 be the distance, in units of length on the graph paper, 
corresponding to the full-scale value of y; let £ 0 be the distance, in the same units of length, but in the 
time direction, corresponding to the t of the system. Define a normalized radius of curvature 

(7.13-3) A^in = A mm T7 0 /{ 0 ■ 

Then fidelity of dynamic response is likely if Am in is large compared to r, and Eq. (7.13-2) may be used. 


In the general case, if the response y(t) is known as a continuous, differentiable function of time, the 
value of Y(t) that produced the response is obtained by performing the operations indicated in Eq. (7. 1-1): 

(1) At any point (r^yj, determine (dy/dt)j. 

(2) Multiply ( dy/dt)i by t. 

(3) Add the product to y,-. This sum is y(r f ). 

In particular, wherever dy/dt = 0, the quantities Y and y are equal (but their slopes are not). 
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The procedure is applicable even when r varies with t or y , provided that r is known. It is necessary 
only to use the appropriate value of r in step 2. [Note N7.13] 

The requirement of differentiability implies that the procedure may not be applied at sharp corners in y (0- 

7.14 Damping of a second-order system. Second-order systems fall into one of several classes that are 
distinguished by the value of the fraction of critical damping. 

(1) If 0 < f < 1, the system is termed underdamped and is capable of an oscillatory transient 
response. Such a system is noticeably different from a first-order system, which is incapable of an oscillatory 
transient response. 

(2) If f = 0, the system is termed undamped. If disturbed, it would enter into an oscillation, at 
frequency /„, that would never die out. [Note N7.14] 

(3) If f > 1 , the system is termed overdamped and is incapable of an oscillatory transient response. 
As f increases beyond unity, the system more closely begins to resemble a first-order system. 

(4) If f > > 1, the system behaves substantially like a first-order system. 

(5) If f = 1 , the system is termed critically damped . The condition represents the transition between 
the underdamped and the overdamped states. The transition is quite gradual as far as the system’s physical 
behavior is concerned, although there may be distinct changes in the mathematical descriptions of that 
behavior. The value f = 1 is the smallest value of f at which transient oscillatory responses become impossible. 

The behavior of an underdamped system is usually best described in terms of the parameters T n , /„, oj*. 
The behavior of an overdamped system is usually best described in terms of r. The relationship among 
these parameters is given by Eq. (7.1-3). 


i*> > l i o < 3 



Figure 7. 15.— Response of a second -order system to a step change. 


7.15 Response of a second-order system to a step change . (Fig. 7.15) If 


(7. 15-la) 

1 

It 

for t > 0 

(7.15-lb) 

Y = y=Y 0 ; y = 0 

for t = 0 

then 




(A) If f = 0, y will start out at zero slope, approach Y u overshoot, and then oscillate sinusoidally 
about Y x with amplitude (Y x — y 0 ) and frequency/*. 

(B) If 0 < f < 1, y will start out at zero slope, approach Y { , overshoot, and then oscillate about 
Y i with continually decreasing amplitude. [Note N7.15.1] 

If the amplitude of the first overshoot beyond Y x is denoted by A 1/2 (Fig. 7. 15) and succeeding overshoots 
by A*, where n denotes an integral number of complete oscillations if n is an integer, these oscillations 
have the following characteristics: 
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(1) The period of the oscillations is T n /a 
where 

(7.15-2) a = V|i — r 2 1 ■ 

(2) The logarithm of the ratio (> 1) of the amplitudes of two oscillations m cycles apart is given by 
(7.15-3) ln(A„/A n+m ) 

where 

(7.15-4) 6 = 2ir{/c< 

is the logarithmic decrement. It is the logarithm of the ratio of the amplitudes of successive cycles. 

[Note N7. 15.2] 

(3) The envelopes of the positive and negative peaks of the oscillations are a pair of symmetrical, 
approximately exponential curves having apparent time constants 

(7.15-5) T/(2f 2 ) = T„/(2ir£) . 

(4) If f < 0.6, the time taken for |y — Y\ | to become less than 10, 5, or 2 percent of \Y 0 - Y ] | is 
approximately T n /( 7rf), 1.5 T n /(i rf), or 2T n /(>f), respectively. 

(5) Qualitatively, the following characteristics of the oscillations are noteworthy: 

(a) The period of the oscillations increases as f increases. 

(b) The decay of the oscillations is more rapid as f increases. 

(c) The time for |y — Y x | to become less than a given small percentage of the initial step 
|K 0 — Y ] | becomes shorter as f increases. 

(C) If f is increased to unity, y will start out at zero slope, pass through a point of inflection (/ 2 -JV 2 ) - 
and thereafter approach Y { in the manner of an exponential curve with time constant slightly higher than 
r. The response is aperiodic, with no overshoot. 

The excursion from Y 0 to Y\ will be almost 99 percent complete after a time interval T n . [Note N7. 15.3] 

This time interval is shorter than it would be for any other value of f. 

(D) If f is increased beyond the value of unity, the response will start out at zero slope, pass through 
a point of inflection (f 2 ,y 2 )> an d thereafter approach Y x in the manner of an exponential curve with time 
constant slightly higher than r. The value of t 2 decreases (from r/2 at f = 1) as f increases, and approaches 
zero as f — 00. The value of y 2 approaches Y 0 as f — 00, [Note N7.15.4] 

Practically, for f 2: 1 , the response is a simple exponential function with time constant r, once the inflection 
point has been passed. [Note N7.15.5] Consequently, if f > > 1, the response is substantially that of the 
basic, first-order system with time constant r. 

7.16 Response of a second-order system to an exponential change. (Fig. 7.16) The response treated 
in this section is representative of the behavior of 

(i) a galvanometer connected to a thermocouple, or 

(ii) a probe microphone wherein a microphone or pressure gauge must be placed at the far end of 
a tube because of space limitations or environmental considerations. 

If the forcing function is 

(7. 16-la) Y = K, - ( r, - Y 0 )e _t/x for t > 0 
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Figure 7. 16.— Response of a second-order system to an exponential forcing function. 


(7. 16- lb) y = V = y 0 ; y = 0 for t = 0 

then y will start out at zero slope and approach Y in one of two ways: 

A. If f < 1, the response will be oscillatory; if f * 0, the oscillation amplitude will decrease with 

time. 

B. If f > 1, the response will be aperiodic and monotonic, and will approximate an exponential 

shape after it has passed through a point of inflection. 

(i) If (X/t) > > 4 f 2 , the time required for the transient to die out, for all practical purposes, is 
short compared to X. The response y will then approach a steady-state condition in which it 
lags y by a constant time interval given by 

(7.16-2) time lag = -X ln[l - (r/X) + r 2 /(4f 2 X 2 )] 

* - X ln(l — t/X) 

(ii) If (X/t) > > 4f 2 and, in addition, X > > t, this time lag is substantially equal to r and the 
later portion of the response is similar to that of a first-order system, as shown in Fig. 7.8. 

(iii) If (X/t) < < 4f 2 and X < < t, the response y will be similar to the response to the step change 
treated in Sec. 7.15 except that the response will be displaced by an additional time lag whose 
magnitude is on the order of X. [Note N7.16] 



Figure 7. 17.— Response of a second-order system to a ramp. 
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7.17 Response of a second-order system to a ramp. (Fig. 7.17) If 


(7. 17-la) 

Y = K 0 + at 

for t > 0 

(7.17-lb) 

O 

II 

II 

II 

for t = 0 


then the response y will start out at zero slope and approach the steady-state condition 
(7.17-2) y=Y 0 + a(t-r) . 

Thus, the asymptote is a ramp parallel to the forcing function but lagging behind it by the constant amount r. 

The approach to the steady state is through a transient that is oscillatory if f < 1 and is aperiodic and 
monotonic if f > 1 . The complete response is given by 

(7.17-3) y = Y 0 + a[t-r{ 1 — A)] 

where A is a dimensionless function of f and of tjr (or t/T n ) and represents the fractional amount by which 
the time lag differs from t. 

(i) If 0.2 <f< 1, the value of A is less than 0.10, 0.05, or 0.02 after a time T n /(2£), 2T n /(3f). 
or 5T„/(6f), respectively. 

(ii) If f > 1, the value of ,4 is less than 0. 10, 0.05, or 0.02 after a time 2.37, 3r, or 4 r, respectively. 
[Note N7. 17] 

The response treated in this section is representative of the behavior of a pressure gauge or manometer 
used to make a steady, continuous traverse of a region in which there is a linear pressure gradient. 


Example 7.17 Suppose that a pressure gradient is to be surveyed with a critically damped pressure 
gauge having a time constant of t seconds, to be moved at a velocity V. The inaccuracy of pressure 
indication p due to the method of surveying is not to exceed 0.1 percent. The a priori estimate of 
the magnitude of the pressure gradient is 1 percent/cm. 

Then the value of a in Eq. (7.17-1) is 

a = (0.01 p/cm)V . 


The steady-state correction is 


ar - (0.01 p/cm)VT . 

If this correction is to be acceptably small it is necessary that 

(0.01 p! cm)kY < 0.001 p 


or 


V < (0.1 It) cm/s 

If this traverse velocity is unacceptably low, a higher traverse velocity may be used, provided that 

(1) the true pressure at time t is taken as the pressure indicated at time (r + r), and 

(2) at least about 37 seconds have elapsed after the initiation of the traverse. 
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Figure 7. 18 — Response of a second-order system to 
a quadratic forcing function. 


7.18 Response of a second-order system to a quadratic forcing function . (Fig. 7.18) If 


(7. 18- la) 

Y = r 0 + to 1 

for t ^ 0 

(7. 18- lb) 

0 

1 ! 

•>, 

£ 

11 

11 

for t = 0 


the response y will start out at zero slope and approach a steady state represented by a parabola having 
the same latus rectum as the forcing function but with vertex displaced to the coordinates ( r t y b ), where 

(7.18-2) y b = b{r 2 -2/u 2 n ) =b[r 2 - T 2 n /(2* 2 )] . 

The steady-state condition is represented by 

(7.18-3) steady-state y = b(t - r) 2 + br 2 [ 1 - (2f 2 ) -1 ] . 

This steady-state response resembles that of the first-order system (Sec. 7.10), differing from it only in 
the position of the vertex, which no longer lies on the original forcing function but is displaced from it 
by a vertical distance br 2 /( 2f 2 ). 

If f = 1/V2, the steady-state y lags behind Y by a constant time interval equal to r. 

If f ^ 1/V2, the time lag approaches 1 as time increases. 

The approach to the steady state is oscillatory if f < 1 and is aperiodic if f > 1. [Note N7.18] 

If 0.2 < f < 0.8 , the time when the response will differ from the steady-state value by less than 
10, 5, or 2 percent of that value will be on the order of TJ 2, 5T n /8, or 3T n /4, respectively. If f £ 0.8, 
the corresponding time will not exceed 2.2r, 2.5 r, or 3. Or, respectively. 

The response treated in this section is representative of the behavior of an impact-plate flowmeter (wherein 
the dynamic pressure of a fluid is converted into a mechanical force) or of a dynamic pressure transducer 
used to make a steady continuous traverse of a region in which there is a linear fluid-velocity gradient. 

7.19 Response of a second-order system to an impulse . If a second-order system is subjected to a step 
change of very high magnitude and very short duration such that the product of duration and magnitude 
is a finite quantity I, termed the impulse , then the response will reach a maximum value in a time dependent 
on the natural period of the system and then drop back to its initial value in a manner identical to the response 
to a step change. 

The behavior is representative of the ballistic galvanometer, which is usually used in the critically damped 
condition, or of the ballistic pendulum. The latter device is convenient for measuring the impulse of an 
explosive charge or of a pulsed rocket. 
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If 

(1) the system is initially such that 
(7.19-1) y = Y 0 ; y = 0 , 

(2) the pulse duration At is so short that 

(7. 19-2a) At < < T n / (2n) if f < 1 

(7. 19-2b) At < < {TJ {lit) if f > 1 , 

(3) the pulse amplitude (Y x - F 0 ) is so large that 

(7.19-3) I=(Y { - Y 0 )*At , 

then 

(1) the response will reach a maximum value given by 

(7. 19-4a) y m - r 0 = /w„exp[-(f/a)sin _1 a] if f < 1 

(7. 19-4b) y m ~ Yq = Iu>„t ~ 1 if r= 1 

(7.19-4c) y m - y 0 = /aj n exp[-(f/a)sinh _l a] if f>l • 

(2) This maximum value will be reached at a time t m given by 

(7. 19-5a) f m = (T n /2ir)r' if 

(7.19-5b) t m = (r„/2ir)(sinh _l a)/a 

- (T n /2n)r m if r>i • 

(3) Thereafter the step-change response equations of Sec. 7.15 will apply, except that / in those 
equations must be replaced by (t — f m ). Amplitude ratios and oscillation periods, when f < 1, 
remain as in Sec. 7.15. [Note N7.19] 



Figure 7.20. 1 .-Steady-state response of a second-order system to a sinusoid. 


7.20 Response of a second-order system to a sinusoid . Only the steady-state response is worthy of 
consideration here [Note N7.20.1]. The application of a steady, sinusoidal forcing function 

(7.20-1) Y = T, sin(cor) = Y ] sin(2x/r) = Y x sin(2irt/T) 

will result in a sinusoidal response of the same frequency but with different amplitude and with lag in time 
and phase (Fig. 7.20.1). This response is represented by 
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(7.20-2a) 

y = AY\ sin(cjf — <p) 

(7.20-2b) 

= AY; sin[w(t - X)] 

where 


(7.20-2c) 

A = 1(1 -r 2 ) 2 + 4fr 2 }- 

(7.20-2d) 

<p = tan"'[2fr/( 1 - r 2 )] 

(7.20-2e) 

X = ip/u = <p*T/(2ir) 

(7.20— 2f) 

r = =///„ = TJT 

The quantities of interest are 


the frequency ratio r t 

the amplitude ratio A, 

the phase-angle 
the time lag X. 


The quantities A and as functions of f and r, are shown in rectangular coordinates in Fig. 7.20.2, 
and in polar coordinates in Fig. 7.20.3. 

The following features of the response are distinctive: 

A. If f = 0, the value of A is 
(7.20-3) A = (1 - rV 1 

On rectangular coordinates, A (r) resembles a quadratic parabola, when r < < 1 . The amplitude ratio becomes 
infinite when r= 1. [Note N7.20.2] 

B. If 0 < f < 1/V2, 

(i) the amplitude ratio A has a maximum at a frequency f r , termed the resonant frequency and given 
by 

(7.20-4) / r =/„V(l - 2f 2 ) 

(ii) the magnitude A r of the amplitude ratio at the resonant frequency is 
(7.20-5) A r = (2fa)-' 

where 

(7.20-6) ct = V 1 1 — f 2 1 

(iii) the locus of these maxima for all values of 0 < f < 1/V2 is given by the curve 
(7.20-7) A r = (1 - r 4 )- ,/2 
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Figure 7.20.2— Amplitude ratio A, time lag X, and phase angle <t> of a second-order system's 
reponse to a sinusoid. 


(iv) for any one curve for which f < 0.2, there are two values of r at which A - 1 = (A r - 1 )/2. 
The difference A r m between these two values is termed the “bandwidth at half maximum” 
[Note N7.20.3]. Its relation to f is given by 

(7.20-8) f * Ar 1/2 /[2V3)(l - 0.84Ar I/2 )] A r, /2 < 0.4 

(v) if “fidelity” of response implies that A shall be near to unity, this condition is more likely to 
be approached over a very wide range of r when 0.5 < f < 0.6. Thus: 


0.85 < A < 1.15 

for 

0 < r < 1.14 

when 

o 

II 

0.95 < A < 1.05 

for 

0 < r < 0.85 

when 

SO 

o 

II 
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Figure 7. 20. 3. —Amplitude ratio A versus phase angle $ of response to a 
sinusoid by a second-order system. 


If f = 1/V2, then both dA/dr = 0 and d 2 A/dr 2 = 0 at r = 0, implying maximum fidelity of response 
when r < < 1. However, there is a 5-percent drop at r = 0.58 and a 15-percent drop at r = 0.79 . 

Thus, for any preassigned limit of error in amplitude response there may be found an optimum value 
of f such that the error remains within the limit over the widest possible range of r. However, the selection 
of f must also be guided by the fact that a random uncertainty in f also exists, either because of imperfect 
knowledge or because of imperfect control. Therefore, in practical situations, any curve in Fig. 7.20.2 
must be replaced by a band representing the uncertainty in knowledge or control of 
In the extreme case when f is unknown except for the knowledge that the system is underdamped, then 

(7.20-9) (1 +r 2 r' < -4 < (1 -r 2 ) -1 . 

The left-hand limit represents the case of critical damping, f = 1; the right-hand limit represents the case 
of no damping, f = 0. 

C. When f > 1/V2, there is no resonant frequency. The response drops monotonically as r increases 
but there is an inflection point. 1 

The drop with increasing r is more abrupt as f increases. As f the response approaches that of a 
first-order system. 

D. For all values of f, the lagging phase angle ip changes from 0 at r = 0, to *72 at r = 1, and 
approaches x as r — 00 . The value of f affects the manner of transition between these fixed points or limits. 

For all values of f, the time lag X changes from 0 at r = 0, to 774 at r = 1, and approaches 772 as 
r — 00 . The value of f affects the manner of transition between these fixed points. 

For f = 1/V2, both phase lag ip and the ratio \/T vary virtually linearly with r over the range 0 < r < 1. 
Consequently, there will be little or negligible distortion of a complex wave shape. 


'The inflection point occurs at r * l when f * 1/V2, at r = 1/V3 when f * 1, and very nearly at r = l/[2\/(2f 2 - 1)] when 

r*2. 
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Figure 7.20.4.— Amplitude ratio of a second-order system's response to a 
sinusoid. 


E. The use of a logarithmic plot of A versus ojt further clarifies the nature of the response 
(Fig. 7.20.4). 

(i) For underdamped systems, the line 
(7.20-10) A = l/(wr) 

is the locus of the maxima of the response curves. The abscissa of any maximum, from 
Eq. (7.20-4), is 

(7.20-11) wr = 2fV(l -2?) 

The ordinate at the maximum is given by Eq. (7.20-5). 

(ii) For underdamped systems, as cor becomes much larger than unity, the slope of A ( wr) approaches 
—2 (“12 dB per octave”). 

(iii) For overdamped systems, if ojt becomes much larger than unity, but remains much larger than 
4f 2 , then A(ut) becomes approximately equal to 4f 2 /(ojr) 2 so that the asymptotic slope of 
A(q)t) is -2, the same as that of an underdamped system. However, if f — oo but 
4f 2 > > ut > > 1, the system begins to resemble a first-order one, with A(o*t) « 1/(ojt). 

F. A polar plot with coordinates (A,tp) is shown in Fig. 7.20.3. All points lie in the first two negative 
quadrants [Note N7.20.4]. Contour lines of f = constant and r = constant are shown. 

Contours for f > 3 are almost indistinguishable from the semicircle which represents the polar plot for 
a first-order system (Fig. 7.12.3). 

Contours of r = constant (r ^ 1) are semicircles passing through the origin and having their centers at 
A = 0.5/|l — r 2 1 and <p = 0 or x. 

G. The amplitude of yf {u n Y \ ) is rA. This amplitude has a maximum value of l/(2£); the maximum 
occurs at r = 1 , regardless of the value of f . 
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7.21 Response of a second-order system to any forcing function. If any arbitrary forcing function can 
be approximated as a sum of the forcing functions treated in Secs. 7.15 to 7.20, the steady-state solution 
is the sum of the steady-state solutions for the component forcing functions. However, the transient solution 
cannot be deduced by any simpler mathematical procedures than by the solution of Eq. 7.1-2. 

The converse problem is of more common practical interest: given an instrument indication y(t) of arbitrary 
wave shape, to deduce the forcing function Y(t ) that produced it. Theoretically, the same technique as 
described in Sec. 7. 13 should be usable; one adds, to the indication y(t), corrections that are proportional 
to the first and second derivatives of y(r), in accordance with Eq. 7.1-2, thereby deriving the forcing function 
Y(t). In practice, this correction procedure has rarely been effective, because of the combined effect of 
errors caused by the process of double differentiation, the uncertainty or inconstancy of f, and the presence 
of noise. 

Under favorable circumstances, in a time interval ±A t surrounding a particular value y(t x ), the value 
of Y(t x ) may be deducible from the information in Secs. 7.15 to 7.20. Such favorable circumstances may be 

(1) The instrument is sufficiently damped so that f exceeds 0.5. 

(2) The principal fluctuations occur at frequencies well below /„. 

(3) Adequate time has elapsed for decay of the transient that originated when the forcing function 
was first applied. 

(4) Abrupt changes in the forcing function, as revealed by the recurrence of damped oscillations, 
are absent. 

Under such circumstances, if the indication in the vicinity of yU*) is oscillatory, the amplitude and time 
lag corrections of Sec. 7.20 may be applied to deduce Y(t x ). If the indication in the vicinity described by 
At * 3r appears monotonic, the assumption that there is a time lag of magnitude r, as suggested by 
Secs. 7.17 and 7.18, may provide a closer estimate of Y(t x ). 

In the absence of identifiable oscillations, a test that will indicate whether y(r) is likely to be a reasonably 
faithful representation of Y(t) is provided by a procedure similar to that outlined in Sec. 7.13: 

The radius A min (in units of time) of the smallest radius of curvature of y(r) is determined, and the 
normalized radius of curvature 

(^•21-1) A min — A mm tj o/^Q 

is computed, where tjq is as defined in Sec. 7.13 but here represents the distance on the graph paper 
corresponding to T n . Then fidelity of response is likely if A* min is large compared to T n . 


If a dynamic system has little damping (say, f < 0.2), the natural oscillations (at period close to T n ) that 
appear in y(t) may obscure the representation of Y(t) even when T(/) varies slowly. In such case, a curve 
may be drawn through the midpoint of every half cycle of the oscillations in y(r), this median curve may 
be treated as an alternative “faired curve/’ and its A' min may be determined. If A^ in > > T n , this faired 
y(r) is likely to be a reasonably faithful representation of /(/). 

7.22 Characterization of the order of a system. The basis upon which a system is designated “first order” 
or “second order” is purely a pragmatic one. If the behavior of a system is indistinguishable from that 
of a first-order system, it may be designated and treated as a first-order system, even though it is known 
to contain elements that, in a strict mathematical sense, make it a second-order system. 

In many borderline cases, such as overdamped second-order systems, large savings in analytical or 
experimental effort may be realized by treating the system as one of the first order, with an acceptably 
small loss in accuracy. 

Example 7.22a. Any real electrical circuit containing a resistor and a capacitor necessarily contains 
some residual inductance in the connecting wires and in the lumped elements. At lower frequencies, 
the inductive reactance may be kept negligible and its presence need not prevent consideration of 
the circuit as a first-order system. On the other hand, at sufficiently high frequencies, the inductive 
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reactance of even a high-quality capacitor may become a significantly large component of the total 
reactance, so that the system must be treated as a second-order one. 

Example 1.22b. A thermometer bulb with distributed mass and finite thermal conductivity constitutes 
a system of higher than first order. In fact, at sufficiently high frequencies of temperature fluctuation, 
the system must be described by the partial differential equation (Fourier’s equation) of heat conduction, 
which represents the limiting condition of a series of first-order systems in tandem, each subjected 
to the forcing function generated by its predecessor. Nevertheless, for many designs and applications, 
no appreciable error is made by treating the bulb as having lumped mass and infinite thermal 
conductivity, and hence as being a first-order system. 

7.23 Instrument selection to provide discrimination in frequency response . If an instrument is to be 
used to indicate variations in the magnitude of a forcing function Y(t), the indications may be obscured 
by the presence of variations in Y(t) that occur at frequencies much higher than the frequencies of the 
variations that are of principal interest [Note N7.23]. It is therefore desirable to select an instrument that 

(i) will provide reliable indications of the magnitude of low-frequency components of the input signal, 
and 

(ii) will have minimal response to high-frequency components of the input signal. 

Solution of this problem requires the instrument user’s intuitive perception, a priori , of 

(1) what frequencies might be present in the signal Y{t ), 

(2) what frequencies are of principal interest in the current application of the instrument, and 

(3) what frequencies are not of interest. 

If frequencies in category 3 are much higher than those in category 2, it may be possible to select an 
instrument whose response at the higher frequencies is much poorer than the response at the lower frequencies. 
Generally, the optimum choices are these: 

(a) If the measuring instrument is essentially a first-order system, its time constant r should not be 
much shorter than the value that would just produce an acceptable measurement error. 

(b) If the measuring instrument is a second-order system, its fraction of critical damping f should 
lie in the vicinity of 0.6 to 0.8 and its natural frequency /„ should be not much higher than the 
value that would produce an acceptable error at the highest frequency /that is of principal interest. 

The need for these optimum choices may not be important when 

(i) high-frequency components of Y(t) are not so large as to obscure materially the components of 
principal interest, or 

(ii) appropriate reduction (filtering) of high-frequency indications can be performed during the data- 
reduction process. 

At the other extreme, the optimum choices indicated above are essential 

(i) when an almost undamped mechanical instrument may be subjected to frequencies so near to its 
resonant frequency that response amplitudes will be destructively large or so large that the 
calibration or elastic properties (Sec. 5.6) of the instrument will be impaired. (Subsequent electrical 
filtering cannot prevent such impairment.) 

(ii) when an almost undamped electrical circuit may be subjected to frequencies so near to its resonant 
frequency that saturation of amplifiers may impair the ability to represent correctly the amplitudes 
at the frequencies of principal interest. 

Example 7.23 a. Structural loads developed in airplane landing gear are often studied by measuring 
the acceleration of the airplane, treated as a single lumped mass, by use of an accelerometer mounted 
near the center of gravity (c.g.) of the airplane. The c.g. acceleration is usually less than 1 g 
(1 g * 10 m/s 2 ) and is imposed at an equivalent frequency on the order of 1 Hz. Any accelerometer 
attached to the airplane structure may also be subjected to local structural vibrations on the order 
of 1 g, but at frequencies of 20 to 1000 Hz. The measurement of interest would therefore be impaired 
severely if the accelerometer had high fidelity at all frequencies. However, an accelerometer with 
f = 0.7 ± 0.1 and/ rt * 5 Hz would provide more than a 10-fold attenuation in indication at high 
frequencies and less than 1 -percent error at 1 Hz. 
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Example 7.23 b. A 10-Hz signal derived from a 10 000-ohm source is to be observed on a cathode- 
ray oscilloscope, when a 60-Hz hum signal of comparable magnitude is superposed on the 10-Hz 
signal. A l-/xF capacitor across the input terminals of the oscilloscope will produce a 15-percent 
systematic reduction in the desired signal but will produce a 4-fold attenuation of the hum. Application 
of a correction factor of 1.15 to the indicated 10-Hz signal may then result in an acceptably small 
uncertainty in the measurement. 

7.24 Initial study of a dynamically changing phenomenon . When a new, unfamiliar experiment or test 
is undertaken, an appropriate first step is to obtain a continuous oscillographic record of the variation with 
time of the principal physical variables that are of concern. High measurement accuracy is not necessary, 
but the measurement system should have reasonably good fidelity in the frequency band that is of interest. 
(A response like that of a slightly underdamped second-order system would be appropriate.) Visual inspection 
of the record will then reveal signal frequencies and rates of change that are present, and their relative 
magnitudes. One may then determine whether there are present, in significant magnitudes, 

(1) power-line frequencies, or their multiples, that may be attributable to hum; 

(2) oscillation frequencies that may be attributable to slightly damped instruments or transmission 
systems; 

(3) oscillations, at higher frequencies outside of the band of interest, that are large enough to obscure 
important wave patterns at lower frequencies; 

(4) spikes or pulses so large that there is danger of damage to instruments or to the validity of acquired 

data; 

(5) signal frequencies truly representative of the phenomenon to be studied. 

An indication of the probable wave shape of the signal will also be obtained. This diagnosis can then 
guide the application of appropriate remedial measures that appear desirable (including limitation of the 
frequency-response band), and can provide a clearer a priori definition of static and dynamic accuracies 
that appear necessary in subsequent experiments or tests. 


NOTES FOR CHAPTER 7. 

N7.4 The clearest way to describe the technique of conversion of a higher-order differentia] equation to 
a combination of first-order and second -order equations is to assign the symbol p to the operator d!dt ~ and 
to treat p as an algebraic quantity, so that 

(N7.4-1) d n y/dt n =p n y . 

Thus, Eq. (7.1-1) may be written as 

(N7.4-2) r 9 py + y = Y 

so that, symbolically, the solution for y may be written as 

(N7.4-3) y = Y/(rp + 1) . 

Similarly, Eq. (7.1-2c) may be written as 

(N7.4-4a) [(1 /u 2 „)p 2 + (2f /w n )p + 1 ]y = Y . 

Here, there are two cases to consider: 

(1) If f ^ 1, the equation 

(N7.4-5) (1 /u 2 „)p 2 + (2 r/«„)p +1=0 
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has >, positive, real roots (if f = 1, the two roots happen to be equal). If these roots are designated 1 !t x 
and l/r 2 , Eq. (N7.4-4a) may be written as 

(N7.4-4b) (r,p + 1 ) (r 2 p + 1 )y = Y 

so that, symbolically, one may write 

(N7.4-4c) (r 2 p + 1 )y = Y/(r { p + 1) 

or 

(N7.4-4d) y = • - • Y . 

r 2 p +1 r,p + 1 

Consequently, in this case, solution of the second-order Eq. (N7.4-4a) may be obtained by first finding 
the response V| of a first-order system with time constant T t to a forcing function Y and then finding the 
response y of a first-order system with time constant r 2 to a forcing function y x . The order in which the 
quantities ( + 1) and {i^p + 1) appear is immaterial. 

(2) If f < 1, Eq. (N7.4-5) has two conjugate, complex roots. A formulation in terms of real parameters 
like t j and r 2 is not possible, and one can only write 

(N7.4-6) y = W(l/^)p 2 + (2 t/u n )p + 1] . 

Turning now to the higher-order differential equation which may be written symbolically as 

(N7.4-7) (a n p n + a n _ } p n ~ [ + . . . + a 2 p 2 + a^p + l)y = Y , 

we note that the equation 

(N7.4-8a) a n p n + a„_\p n ~ l + . . . + a 2 p 2 + a x p 4-1=0 

may have k real roots and m pairs of conjugate complex roots, such that 

(N7.4-8b) k + 2m = n 

Equation (N7.4-7) may then be factored and written as 
(N7.4-9a) U ] U 2 y=Y 

where II] is the product of factors F t of the form 
(N7.4-10) F, = (t^ -hi) (/ = 1,2,...,*) 
and Il 2 is the product of factors G } of the form 
(N7.4-11) Gj = p 2 /o) 2 nj + {2£j/w„j)p +1 0 = 1.2 

so that 

k m 

(N7.4-9b) n, = Yl F t ; U 2 = G ; 

1 1 

Thus, the solution of Eq. (N7.4-7) may be represented symbolically as 
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(N7.4-12) 


1 1 

v = 

• G, C 2 


1 



1 1 1 

l • • • • 

^ ■" F k 


Y 


where the order of terms in the product is immaterial. This represents a tandem or serial sequence of 
operations. 

Equation (N7.4-12) may also be written as 



where the as are algebraic functions of the r,’s, the /3’s are algebraic functions of p, the £’s, and the c*j„/s, 
and where the order of terms in the sum is immaterial. This represents a parallel group of operations. Because 
of the presence of p in the j3’s, this equation does not lend itself to an intuitive perception of system behavior, 
or to easy simulation by analog methods, unless U 2 is unity (i.e., unless all the roots of Eq. (N7.4-8a) 
are real). 

The use of the operator p, which is treated in most elementary texts on differential equations (where the 
preferred symbol is usually D), forms the basis of Heaviside’s operational calculus. For easier mathematical 
solution of differential equations, the modern approach is the use of the Laplace transform (Ref. 7-1). 
However , the Laplace transform does not provide a description of the physical situation that permits easy 
interpretation and deduction of the system’s response merely by inspection. The differential equation itself, 
or the symbolic use of the operator p or D, does provide such understanding more easily. This point has 
been made by Trimmer (Ref. 7-2), who also provides a more complete exposition of the subject of dynamic 
response. 

N7.7 Equation (7.1-1), with right-hand term zero, occurs in the description of radioactivity. When a given 
mass of a radioactive nuclide emits particles, the mass becomes depleted. Since the number of particles 
emitted per unit time is proportional to the remaining mass, one may write 

AN = -(\/r)N*At 

to indicate that the drop AN in emission rate will be proportional to the elapsed time At. If At is so large 
that the random error in measuring N is acceptably small and if the measurements are recorded over a 
sufficiently long total time scale that the concepts of the differential calculus may be applied, it may be 
justifiable to write 


r*{dN/dt) + iV = 0 . 

The half life t, / 2 of the particular disintegration is the time interval in which N would become one half 
of its value at the beginning of the interval. Its relation to r is 


Tin = r In 2 . 

N7.8 As indicated in Sec. 7.4, the response y may be considered either (a) as the response of an instrument 
with time constant r to an exponential forcing function with time constant X or (b) as the response of an 
instrument with time constant X to an exponential forcing function with time constant t. 

If X is chosen as the larger of the two time constants, and we define 

(N7.8-1) m = X/r m > 1 , 

the following relations, illustrated in Fig. 7.8, serve as convenient indicators of the nature of the response. 
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(1) The coordinates of the inflection point are given by 


(N7.8-2a) 

t 2 = (X In m)/(m-\ ) 

X > 7 

(N7.8-2b) 

u = x 

X = 7 

(N7.8-3a) 

to - >o)/ ( i^i - y 0 ) = i - (« + 

X > 7 

(N7.8-3b) 

= 0.26/m 1/4 



(2) The tangent at the inflection point intersects y = Y 0 at t h where 
(N7.8-4) t j = rmf (m + 3) m ss (X/r) > 1 

(3) Over the range of possible values of X/t, the value of y - Y 0 at t = X + r will lie between 37 
and 41 percent of K] - Y 0 ; 

(4) At times r = n(\ + r) the value of y - Y 0 will be not less than the fraction of T, - Y 0 tabulated 
below: 



n =2 

3 

4 


fraction = 0.86 

0.95 

0.98 0. 

N7.9 If 




(N7.9-la) 

Y= y, + at 

for 

t > 0 

(N7.9-lb) 

Y= Yo.' y = Y 0 

at 

t = 0 


then the change in Y is a combination of a step change of magnitude ( Y x - Y 0 ) and a ramp Y 0 + at. The 
response is 

(N7.9-2) y = Kj + a(t - r) -f (Y 0 - Y x 4 - ar)e~ tfr 
and the asymptotic steady state is 
(N7.9-3) T, +a(t- t ). 

N7.ll A pulse of triangular shape and very short duration At may be defined by the equations 


(N7. 11-la) 

Y — Y 0 = 2Y,t/At 

for 

0 < t < At/2 

(N7. 11-lb) 

Y- y 0 = 2/1(1 - t/At ) 

for 

(At/2) < t < At < < t 

(N7.11-lc) 

II 

II 

at 

t = 0 

(N7.11-ld) 

II 

for 

t > At 

(N7.11-2) 

(y, - y 0 )*Ar/2 = / 
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An approximation of the response is given by 


(N7.1 1 —3a) 

v - Y 0 = (//2r)[l - cos(irf/Af)] 

for 

0 < t < At 

(N7.1 l-3b) 

v- y 0 = (1/ T )t- Ur 

for 

t > At 


The response closely resembles the response to a rectangular pulse with the same value of / — the peak 
amplitude reached is Y 0 + 7/r and reversal of direction occurs only when y = Y on the descending leg of 
the triangle, at which time t is almost equal to At. 

N7.12.1 If the forcing function is a pure sinusoid of amplitude Y\ and average value Y 0 and the initial value 
of v is y 0 > *h e magnitude of the transient will usually have become less than 15, 5, or 2 percent of 
\Y l + *0 “ yol at a time 2 t, 3 r, or 4 r, respectively, after the imposition of the sinusoid. 

N7.12.2 Capacitors considered to be of high quality may have a dissipation factor smaller than 0.001 at 1 kHz. 

N7.12.3 Inductors considered to be of high quality may have a storage factor larger than unity at 200 Hz, 
where losses are primarily due to wire resistance. At high frequencies, energy dissipation is primarily due 
to eddy currents, and there is additional loss due to distributed capacitance. An inductor considered to be 
of high quality may have a storage factor larger than unity at 1 MHz. 

N7.13 The accuracy achievable by the three-step procedure depends principally on the ability to determine 
( dy/dt) h If y(r) is in the form of a continuous electrical signal, analog differentiation with careful attention 
to minimizing the effects of hum and noise usually is most effective. If the derivative is determined by 
sampling at discrete intervals and assuming that Ay/ At represents dy/dt, then the interval At must be chosen 
carefully: choice of too small an interval leads to excessive random errors; choice of too large an interval 
leads to excessive errors of nonlinearity. 

N7.14 For a completely self-contained system with no external source of power, this condition is impossible 
because it implies perpetual motion. However, it is a convenient limit of behavior and often can be approached 
so closely that the simplifying assumption f = 0 can be made with negligible error. 

N7.15.1 The equation of the response is 

(N7.15-la) y = K, + (y 0 - Y { )a-'t~ ff ' sin (««„/ + 

where 

(N7.15-lb) a = V|l-f 2 |; 0 = {w n ; j = cos-'f . 

N7.15.2 The ratio between the initial amplitude difference Aq = \Y X — Y 0 \ and the First overshoot A l/2 has 
been termed the damping factor and has been used by manufacturers of galvanometer-type instruments to 
describe the damping of such meters. This measure of damping is useful because, in a well-damped meter, 
the second overshoot, Aj, may be imperceptible. 

N7.15.3 The equation of the response is 

(N7. 15-2) j = r, + ( Y 0 - Y ] )f ~ 2,/r ( 1 + 2th) . 

The coordinates of the inflection point are given by 


h = r/2; (y 2 - Y 0 )/(Y l - K 0 ) = 1 — 2/« . 
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The slope at the inflection point is 

2(Y } - Y 0 )/(er) . 

The tangent at the inflection point intersects Y { at a time 37/2. 

The time for the excursion to be almost complete is given in the following table: 


Percent completion 

90 

95 

98 

99 

Multiple of T n 

0.62 

0.76 

0.93 

1.05 

Multiple of 7 

1.9 

2.4 

2.9 

3.3 


N7.15.4 The equation of the response is 

(N7.15-3a) >’ = K, + (Y 0 - ~ 0 ' sinh(2fc*f/r + i£) 

where 

(N7.15-3b) a = V|f 2 — 1 1; 0 = 2 { 2 /t\ ^ = sinh _1 a 

The coordinates of the inflection point (r 2 .3'2 ) are given by 
(N7.15-4a) t 2 = [r/(4fa)] • ln[(f + a)/ (f - a)J 

(N7.15-4b) (y 2 - Y 0 )/(Y> - Y 0 ) = 1 - 2{-[(f - a)/(f + a)] f/<2a) 

The slope of the tangent at the inflection point is (T t — yi)/r . 

N7.15.5 The difference between y as given by Eq. (N7.15-3) and the y given by 

y * ( - >7) (l - exp[(/ 2 - r)/T] j 
is less than 8, 2, or 1 percent when £*= 1, 2, or 3, respectively. 

N7.16 If 

(N7. 16-la) z = (Y x -y)f{Y x - T 0 ) 

(N7.16-lb) or = V|r 2 - 1 1; /3 = 2r 2 /T = « n f=2irf/T n 

(N7.16-lc) m = X/t 
(N 7.16-2a) A = 1 - 2f/(«„X) + (w n X) ~ 2 
= 1 -m~' + (2fm)" 2 

(N7.16-2b) B = (a/f)(l - 4? 2 m)/(\ +2 m - 4?m) 

then the responses to the exponential forcing function of Eq. (7.16-1) are given by 
(N7.16-3) Az = (lM)e _,/X + (Xw„a) -1 e _0 'sin(w n ar + tan“‘fl) if f < 1 

(N7.16-4) A 2 z = f _,/X + (4m 2 )“'[l -4m + (2 - 4m) (//r)]f _2 ' /r iff=l,r*2X 
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(N7. 16-5) z = ( 1 + 2 t/r + 2f 2 /r 2 )e " 2,/T if f = 1 , r = 2X 

(N7.16-6) Az = (lA4)e“' /X + (2mfa) ~ l € “^ f sinh(2 far / t + tanh ~ ] B) if f > 1 

N7.17 The multiples of r quoted for £* > 1 are conservative. They are limits as f — oo. More precisely, if 


(N7.17-1) 

« = V|i- 2 - 1 1; 0 = 2 f 2 /r = = 2xf/7'„ 


(N7.17-2) 

B = 2fa/(2f 2 - 1) ; 


then 



(N7.17-3) 

A = (2fa) _1 € -,3 'sin(aa>„f 4- tan _1 B) 

if r < i 

(N7.17-4) 

A = «- 2,/r (l + t/r) 

if f = 1 

(N7.17-5) 

A = (2ra)" , e" s 'sinh(2rat/r + tanh" 1 B) 

if f > 1 

N7.18 The complete response to a quadratic parabola is given by 


(N7.18-1) 

y/b = (t-r) 2 + C 


where 



(N7.18-2) 

C- r 2 (l-A) + r 2 (A/a)e ~ s 'sin(au„f + tan _l B) 

«/r< i 

(N7.18-3) 

C = r 2 / 2 - (r 2 / 2)e " 2,/T (3 + 2r/r) 

if r = i 

(N7.18-4) 

C=7 2 (l — A) - T 2 (/4/a)e -e 'sinh(2far/T + tanh _1 B) 

if f> 1 


where 


a - V|f 2 - 1 1; /3 = = 2 f 2 /r = 2xf/r„ 

(N7.18-5) /I =(2^)"' 

B= (a/ft(4r 2 - l)/(4f 2 — 3) 

N7.19 The exact equations for the response to a pulse of amplitude ( Y t — T 0 ) and duration At are 

A. For f < 1, 

y m -Y 0 = (T, - Y 0 )e ~ t U 2b - 2e*cos a + 1 ) 1/2 

t m = (w„a) -1 tan -l [e*sin a /(e fc cos a — 1)] 

B. For f = 1, 

Ym- io= O'. - T 0 )f" f (^- 1) 

r m = Ar/(1 -e- 6 ) 
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C. For f > 1 , 

y m -Y 0 = (Yi- r 0 )€ ~ c u 2b - 2e*cosh a + 1 ) 1/2 
t m = (aj„a) ~ l tanh' l [e*sinh a/(e fc cosh a — 1)] 


where 


a = ay**A /; b = i"cd n «Af; c = fa> n f m 

and the impulse 1 is given by 


/= (Ki - y 0 )*Ar 

N7.20.1 The transient response becomes academic because it depends on the phase angle of the forcing 
sinusoid at the instant it is imposed on the system. However, an intuition of the time interval required for 
the transient response to become small is provided by the information in Sec. 7.15 concerning the response 
to a step change. 

N7.20.2 Although Eq. (7.20-3) represents a fictitious mathematical limit, its simplicity makes it useful 
for describing instruments, like silicon-crystal force or pressure transducers, which have very little damping 
and are usually used with r < < 1 . 

The equation also indicates that, if the condition r — 1 is approached closely enough, destructively large 
amplitudes of oscillation may occur. 

N7.20.3 The two values of frequency ratio r, at which A — 1 = (A r — l)/2, are given by 

r \/2 = 1 — 2£ 2 ± x(3 - lx + x 2 ) 1/2 /( 1 + x ) 


where 


x s 2fa = 1 !A r . 

Equation (7.20-8) yields f with a limit of error of 0.003 only when f < 0.2. The difference A r u2 between 
these two values of r 1/2 is useful in several scientific fields. It may be used to describe the sharpness of 
tuning of an electrical circuit or of a mechanical or electro-optical oscillator. With a different mathematical 
definition— the width of the abscissa segment joining two points whose ordinates are one half the height 
of the peak— the “bandwidth at half maximum” may also be used to describe the resolving power of a 
mass spectrometer or optical spectrometer at some stated value of abscissa. 

N7.20.4 In a polar plot of this type (a Nyquist diagram), the number of quadrants occupied by the plot 
is equal to the order of the differential equation describing the physical system (Sec. 7.4). The diagram 
is particularly valuable in describing such systems because they are capable of self-oscillation if, and only 
if, the point (1,tt) lies on or inside of the plot. 

N7.23 If the amplitude of Y is estimated to be Y x and the maximum rate of change of Y that is of interest 
is Y m , then the highest frequency of interest is 

f m = Y m /a*Y x ) . 

Input signals at frequency 4/ m and amplitude Y } / 8 could add 0.5 Y m to the indicated rate of change if 
the fidelity of instrument response y(f) were the same at both f m and 4/ m . 
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CHAPTER 8. EXPERIMENTAL DETERMINATION OF THE 
PARAMETERS OF DYNAMICAL SYSTEMS 


8.0 Introduction . If a system is believed to approximate a first- or second-order system, the imposition 
of a known forcing function, for which the solution is known, may serve to reveal the characteristic parameters 
of the system. Conversely, observation of the nature of the system's response to some simple forcing function 
may serve to establish the extent to which a system may be considered to be a first-order or second-order 
one. Application of the same test to a more complex system may serve to identify one or more simple systems 
contained within the complex one. 

The most convenient forms of forcing function for this purpose are the step, impulse, ramp, and sinusoid. 
An alternative technique, the unit-deflection method, applicable to mechanical systems using springs, is 
a combination of analytical and experimental techniques that permits determination of the natural frequency 
without recourse to dynamic tests. 

The symbol list of Sec. 7.0 also applies to this chapter. 

8.1 Test by imposition of a step or impulse . A step change may be imposed, for example, 

(a) on a mechanical spring, by deflecting it a fixed amount and then suddenly releasing it (“plucking”); 

(b) on a pneumatic system, by pressurizing it to a steady value and then suddenly releasing the pressure 
(“burst-diaphragm method”); 

(c) on a pressure gauge, by use of a shock tube (the step is of short duration, that depends on the 
length of the tube); 

(d) on an electrical system, by connecting or disconnecting a source of steady voltage or current 
(“keying”); 

(e) on a thermometer bulb, by sudden immersion in a well- stirred bath of different temperature; 

(f) on a radiometer, by turning on or off a source of incident radiation. 

An impulse may be imposed, for example, 

(a) on a mechanical system, by the single stroke of a hammer, or by the single bounce of a pellet; 

(b) on a pneumatic system, by the incidence of a shock or detonation wave; 

(c) on an electrical system, by a capacitor discharge with very short time constant (ballistic method); 

(d) on a radiometer, by use of a short-duration, high-intensity flash. 

A. If the response is clearly oscillatory and its decrement is constant (i.e., the ratio between successive 
amplitudes is constant), the system is like the basic second-order system. The value of f may be determined 
from the measured amplitude ratios by use of Eq. (7.15-3) for the logarithmic decrement 6 and by 

(8.1-1) f 2 = S 2 /(6 2 +4 r 2 ) . 

The period of the oscillations may be measured and T n computed as 
(8.1-2) T n = period x V(1 - f 2 ) 
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B. If the response is clearly oscillatory, but the ratio of successive amplitudes becomes closer to unity 
as the oscillation continues, there is appreciable square law damping. (See Sec. 10.4.) The period of the 
oscillations may be measured and assumed equal to T„. The first overshoot A 1/2 may be measured and an 
equivalent f computed as 

(8.1-3) f m = (l/7r)!n(VA„ 2 ) if L < < 1 , 

where Ag is the magnitude of the initial step. One may conclude that the system will act like a linear system 
with f < f m , but additional quantitative deductions are not easily possible. 

C. If the response is only slightly oscillatory, or if there is only one noticeable overshoot, the value 
of f may be determined by use of Eq. (8.1-1), where 

(8.1-4) 5 = 2 ln(Ao/A 1/2 ) . 

However, the determination of T„ by use of Eq. (8.1-2) may be of inadequate accuracy. This accuracy 
may be improved if the damping can be temporarily removed or reduced, thereby permitting use of method 
A above. 

Generally, a system, whose response is so slightly oscillatory that its f and T n cannot be measured 
accurately, can be treated as a critically damped second-order system, with characteristics as described in 
Sec. 7.15 C. 

D. If the response is aperiodic, with nonzero initial slope, and the graph of dimensionless amplitude 
z = O' - Y X )/(Y 0 - T,) versus time is approximately a straight line on semilog paper, the system is 
substantially of the first order and its r may be determined from the average slope of the line in the highest 
decade of z. If the exponential character of the response is evident, the plot on semilog paper may be omitted 
and r determined immediately as the time when z = 1 /e = 0.368 or as the absolute value of the reciprocal 
of the initial slope of z(t) (i.e., the time when the tangent to the response curve at t = 0, z — 1 intersects 
the ordinate z = 0). 

E. If the response to a step is aperiodic, but starts at zero slope, or if the response to an impulse is 
aperiodic after an initial peak has been reached, so that there is a point of inflection in the response curve, 
the system may be considered to be of the second order and to be overdamped. 

Construct the tangent at the point of inflection (the tangent with maximum positive or maximum negative 
slope). Let r 3 be the time at which this tangent intersects the ordinate z = 0. Let t 2 be the abscissa of the 
point of inflection. Then 

(8.1-5) r = r 3 — t 2 . 

If, in response to a step, the point of inflection occurs at z £ 0.9 (implying that f > 3), a simpler procedure 
is adequate: Determine the time t m at which z = 1/2. Then 

( 8 . 1 - 6 ) t = t u2 /0.1 . 

This equation is comparable to 

(8.1-7) r = r 1/2 /ln2 

for a first-order system. 


8.2 Test by imposition of a ramp. If a linear change Y = at is applied to the instrument, the time 
displacement between the forcing function and the asymptotic steady state of the response represents r. 
If the applied ramp change is not accurately linear, such procedure will still yield t if the asymptote is 
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drawn as a curve parallel to the forcing function, since the time lag in response to a quadratic parabola 
also approaches r. 

If the approach to the steady state is oscillatory, the difference between the actual indication and the 
asymptotic steady state may be replotted or measured directly to provide amplitude ratios and frequencies 
in the same manner as was done for the step change. 

8.3 Test by imposition of a sinusoid or of white noise. Particularly for electrical systems, the response 
of an instrument to an impressed sinusoidal signal is a convenient way to estimate the characteristic parameters 
of the instrument. Generally, measurements are made of the amplitude ratio A = y\/Y x between response 
amplitude y\ and impressed amplitude Y\, and of the phase angle <p between the two waves. The procedure 
is usually effective only when it is possible to vary the forcing frequency /over a wide range. This variation 
may be effected by continuously varying the forcing frequency while maintaining constant amplitude Y { 
or else by subjecting the instrument to “white noise, ” as may be developed by a reverse-biased diode 
(Sec. 5.19), wherein all frequencies of interest are generated virtually simultaneously. A spectrum analyzer 
or its equivalent is then required. If a white-noise generator is used, there must be careful confirmation 
that the signal amplitude it generates is the same at all frequencies. 

Usually, for second-order systems, the sweep through the band of frequencies of interest seeks to determine 
the frequency f r at which a maximum value of A is obtained; this value is the amplitude at resonance A r 
(Sec. 7.20). 

A. If a system is known, in advance, to be of the first order, then the time constant r may be determined 
from measurement of either the amplitude ratio A or the phase angle <p when the steady state has been reached. 
Convenient formulas are 

(8.3-1) t = V3/(2t/) when A- 1/2 or <p = 60° 

(8.3-2a) r = l/(2x/) when A = 1/V2 or <p = 45° 

where /is the frequency at which the designated value of A or <p occurs. Equation (7.12-2) may be used 

for other amplitude ratios and phase angles. 

B. If a plot, on log paper, of amplitude ratio A versus frequency / appears to be asymptotic to a line 
of slope - 1 as/increases, the system is very nearly of the first order and Eq. (8.3-2a) may be applicable. 
The value of r is deducible from 

(8.3-2b) r = 1/(2 x/) 

where/., the comer frequency, is the frequency at which the asymptotic line of slope -1 intersects the 
ordinate A = 1. 

C. A polar plot of amplitude ratio A versus phase angle <p, like Fig. 7.20.3, serves to identify both 
f and /. The extent to which the plot lies in the second negative quadrant is a measure of the relative 
importance of the second-order features of the system. 

D. The frequency /at which the phase angle is 90° is also the natural frequency / of a second-order 
system. Only when <p = 90° can / be determined without knowledge of For a first-order system, 
<p — 90° as /— oo. 

E. If appreciable resonance (A r > 1) is detected at some frequency /, then the value of f is deducible 
from A r by the formula 

(8.3-3) (1 - 2f 2 ) 2 = 1 - MA] 

and the natural frequency / is given by 

(8.3-4) f 2 „=fW(l-l/A 2 r ) . 
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F. For slightly-damped systems ( A r > 2), the amplitude ratio A r yields 
(8.3-5a) f * l/(2i4 r ) 

and the “bandwidth at half-maximum” yields f by Eq. (7.20-8), where 
(8.3-5b) A r l/2 * (/] 12 S— 1/2 )/fr 

and f \/2 and /_ 1/2 are the values of/ at which the amplitude is (A r + l)/2. Equation (8.3-5a) determines 
f from an amplitude ratio; Eq. (8.3— 5b) determines f from a frequency-difference measurement, which 
may require high accuracy when the bandwidth is narrow. 

G. If appreciable resonance is detected at several frequencies, then the formulas in E. and F. may be 
applied to each of these, so that the behavior of the entire physical system is understood, even though the 
actual arrangement of its components may not be. 

H. An alternative approach is possible if the derivative y of the instrument indication can be measured, 
rather than the indication itself. By Sec. 7.20 G, the forcing frequency at which y is a maximum (while 
forcing amplitude Y\ is held constant) is also the natural frequency of the instrument, regardless of the value 
of f. Furthermore, if both Y x and y are measured at the resonant frequency, then can also be determined. 
This technique is particularly useful when dealing with electrical signals that can be differentiated easily. 

8.4 Unit deflection method of determining natural frequency . This method is a combination of analytical 
and experimental techniques. It generally involves (a) the analytical calculation of a constant that is 
characteristic of a particular instrument design and (b) the experimental measurement of a deflection. However, 
the characteristic constant could also be determined empirically. 

For a particular design of a simple second-order system there generally exists a relation of the form 

(8.4-1) fid = a constant 

where 6 is an experimentally measured deflection and the constant is characteristic of the instrument design. 
For any mechanical system where 6 is a length, the constant has the dimensions of acceleration. The unit 
deflection method becomes particularly convenient when the constant is proportional to the local acceleration 
of gravity g. The determination of f n then involves no dynamic testing and is not dependent on knowledge 
of the damping. 

A. Linear spring-mass system. 

(i) If a concentrated mass m is supported by a massless spring of spring constant k (Sec. 6. 1), the 
deflection 6 when gravity acts upon the mass is given by 

(8.4-2) 6 = mg/k . 

Consequently, by Eq. (6.1-2a), 

(8.4-3) fl& = g/ (4ir 2 ) 

The deflection 6 is the change in spring deflection after the assembly is turned 90° so that gravity no longer 
acts to deflect the spring. 

The spring may be of any form: helical, cantilever, diaphragm, etc., provided its mass is negligible, but 
6 must represent the deflection of the spring itself (the deflection measured in the determination of k) and 
not of any mechanical magnifying device. 

If f n is in hertz and b is in inches, 

(8.4-4a) fib * 10 . 
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If f n is in hertz and 6 is in centimeters, 

(8.4-4b) f„b rn 25 . 

(ii) If there is no concentrated mass (m — 0) and the deflection caused by gravity is due entirely 
to the distributed mass m 5 of the spring, then 

(8.4-5) f 2 n b = 0g/(4ir 2 ) 

but 13 now depends on the form of the spring and usually has a value between 5/4 and 5/3 (Footnote 1). 

(iii) If the spring has appreciable mass and there is also a mass concentrated at the end of the spring, 
then 13 in Eq. (8.4-5) has a value intermediate between unity and its value in case (ii). 

B. Rotational spring-mass system . For the system treated in Sec. 6.3, if 6 is the angular deflection 
caused by an angular acceleration of 1 radian per second 2 , and f n is in hertz, then 

(8.4-6) f 2 n b = 1/(4tt 2 ) 

This relationship is principally of academic interest, because of the practical difficulty of creating unit angular 
acceleration. 

C. d’Arsonval galvanometer. For a d’Arsonval galvanometer (permanent magnet, moving coil meter), 
possessing a coil resistance R and a constant sensitivity S , where 

(8.4-7) S = (angular coil rotation)/(coil current) 

the comparable relation is 

(8.4-8) / 2 5/V/? = a constant 

The constant depends on the size, shape, and material of the coil and on the flux density in the air gap; 
it is usually determined empirically. The magnitude of this constant is sometimes taken as a figure of merit 
of moving-coil oscillograph design. 


iFor a helical spring or bellows, /3 = 1.50 
For a cantilever of constant cross section, /3 * 1.56 
For a circular diaphragm with simply supported edges, 0 * 1.56 
For a circular diaphragm with clamped edges, /3 * 1.67 




CHAPTER 9. ANALYTIC DETERMINATION OF THE 

CHARACTERISTICS OF DYNAMICAL SYSTEMS 


9.0 Introduction . In Chapter 6, the differential equations describing the behavior of various systems were 
derived by use of Newton’s law for the balance of forces in mechanical systems or Kirchoff s laws for 
the balance of currents and emf s in electrical systems. Other laws, such as those for the balance of power 
or energy, can be used. Some will be illustrated here, because, although they may offer little or no advantage 
for the treatment of simple first- and second-order systems, they become more useful as systems become 
complex. 

Once a differential equation has been derived, simple algebraic manipulation, generally involving 
transformation of the dependent variable to a dimensionless form, may allow reduction of the entire equation 
to a simple canonical form whose behavior is already known. The advantage gained is that inspection of 
the equation and identification of its characteristic constants is sufficient for an understanding of the behavior 
of the system, without recourse to detailed solution of each particular equation. 

The symbol list of Chapter 7 also applies to this chapter. 

9. 1 Power equation . A balance between power input and power output was used to establish the differential 
equation for the thermometer bulb (Sec. 6.12). A similar power balance equation can be used for mechanical 
or electrical systems. 

Example 9.1 a. For the linear mechanical system, Sec. 6.1, at any displacement x, 

work done by external force = J* F^dx = Fx 

energy stored in spring = j* kx*dx = kx 2 /l 

rate of dissipation of energy in oil = (bx) m x = bx 2 
energy of motion of mass = mi 2 / 2 

The power balance requires that 



so that 


Fx = kxi + bx 2 + mxx 

and dividing through by i yields Eq. (6.1-lb). 
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Example 9.1 b. For L, C, and R in series, Sec. 6.8, 

power delivered by external voltage = Ei = Eq 
energy stored in inductance = Li 2 / 2 = Lq 2 / 2 
energy stored in capacitance = q 2 / (2C) 
rate of dissipation of energy in resistance = Ri 2 = Rq 2 
The power balance requires that 


d ( Lq 2 \ d ( q 2 \ 2 

+ Rq 2 = Eq 

dt\ 2 ) dt\2CJ 


so that 


Lqq + qq/C + Rq 2 = Eq 
and dividing through by q yields Eq. (6.8- la). 

9.2 Rayleigh’s method of determining natural frequency . This method is used to compute the natural 
frequency /„ of a system in the absence of appreciable damping. It equates the energy of the system at two 
instances when the energy is particularly easy to calculate, assuming that at both instances the system is 
oscillating at frequency f n [Note N9.2]. 

For a mechanical system, these instances are 

(1) when oscillation amplitude is a maximum and oscillation velocity is zero, so that potential energy 
is a maximum; 

(2) when oscillation velocity is a maximum and oscillation amplitude is zero, so that kinetic energy 
is a maximum. 

For an electrical circuit, these instances may be 

(1) when oscillation amplitude (of voltage, current, or charge) is a maximum; 

(2) when the same oscillation amplitude is zero. 

Example 9.2 a. If the linear mechanical system, Sec. 6. 1 , is oscillating at its natural frequency so that 

x = x m sin(2 irfj) , 

the maximum potential energy occurs when x = x m> i = 0, and is then 

kx 2 /2 = (l/2)x 2 Jc . 

The maximum kinetic energy occurs when x = 0, x = 2icf n x mt and is then 

mx 2 /2 = (1/2)(2t/„) 2 x> . 

Equating these two energies leads to 


(2 t /„) 2 = k/m 


in agreement with Eq. (6, 1— 2a) 
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Example 9.2 b. If the current in a circuit with inductance L and capacitance C in series is oscillating 
at frequency f n so that 
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i = i m sin(2* f n t) 

the energy is stored entirely in the inductor when i is a maximum. The energy is then 

(l/2)Li‘i . 

The energy is stored entirely in the capacitor when the capacitor's charge (the time integral of /) 
is a maximum. The energy is then 


(2C)- l i 2 m /(2irf n ) 2 . 


Equating these two energies leads to 


(2 ir/„) 2 = \/{LC) 


in agreement with Eq. (6.8-2b). 

9.3 Lagrange's equation . This equation is usable in the most general case when there is appreciable 
dissipation (damping). It is derived from the forces, energies, and rate of energy dissipation that exist at 
any instant. Let 

U = kinetic energy of the system 
V = potential energy of the system 
D = rate of dissipation of energy of the system 
LFj = sum of external forces acting upon the system 
x t = independent coordinates on which U , V, and D depend. 

Then, for each independent coordinate x { 


„ d/dU\ BU 3V 1 dD _ 

(9.3-1) - — + — + = LF; 

dt\bxij dx i dx, 2 dx t J 


For mechanical systems, Lagrange’s equation is an equation of forces. For other systems, like electrical 
circuits, the quantities x, and Fj are generalized coordinates and generalized forces , respectively, chosen 
so that they satisfy the condition 

(9.3-2) (generalized coordinate) x (generalized force) = energy . 

Then U is a quadratic function of the generalized velocities (the time rate of change of the generalized 
coordinates) and V is a quadratic function of the generalized coordinates. A separate differential equation 
(9.3-1) is written for each x f and the simultaneous solution of these equations leads to a description of 
system behavior. [Note N9.2] 

Example 9.3 a. In an electrical circuit containing L, C, and R in series (Fig. 6.8), one may take 

F } = E; = q 


so that 


U = Lq 2 / 2; V=q 2 /(2C); D = q 2 R 
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Equation (6.8-la) then follows from Eq. (9.3-1). 

Example 9.3 b. In an electrical circuit containing L, C, and R in parallel (Fig. 6.9), one may take 

F i = 1; jcj = ^F*dt b u 


so that 


U = u 2 C/2; V = u 2 /(2L); D = u 2 /R . 
Equation (6.9-lc) follows from application of Eq. (9.3-1) and the fact that 

u = fi L . 



Figure 9.3.— Dynamic vibration absorber. 


Example 9.3 c. The dynamic vibration absorber , Fig. 9.3, uses a smaller damped spring-mass 
system (m 2f k 2 , b 2 ), termed the absorber , to reduce the vibration of the almost undamped principal 
system {m x , k\, b x * 0) when the latter is subjected to a force F varying sinusoidally at a fre- 
quency / that is near to the natural frequency f nl of the principal system. 

In the absence of damping {b 2 =0), the otherwise high amplitude of oscillation of system 1, that 
occurs when / = /,, can be reduced to zero by making the natural frequency fa of system 2 equal 
to f„\. However, this reduction would be achieved at the expense of two usually unacceptable 
consequences: 

(i) the amplitude of oscillation of system 2 is very high, and 

(ii) system 1 has two other resonance points, one on each side of/ nl , at which the amplitude 
of oscillation of system 1 is very high. 

By introducing the appropriate amount of damping into system 2 and by selecting an adequately 
large value of m 2 , a compromise can be achieved in which the amplitudes of oscillation of both 
systems can be held within acceptable limits over the entire operating frequency range. 

In Fig. 9.3, if x } and x 2 are the respective elongations of springs 1 and 2, from their respective 
positions when there is no vibration, and the symbolism parallels that of Secs. 6.1 and 7.0, 


potential energy of system 1 
potential energy of system 2 
kinetic energy of system 1 
kinetic energy of system 2 
power dissipation in system 1 


= ( 1 / 2 )*,*? 

= (l/2)*2*2 

= (1/2 )m,*? 

= (l/2)iM 2 (i, + x 2 ) 2 
= M l 


power dissipation in system 2 = b 2 x\ 
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Consequently, the dependent variables to be inserted into Eq. (9.3-1) are 

V = (1/2 )k { x] + (1/2 )k 2 x\ 

U = ( 1 / 2 ) w , jc ? + (l/2)m 2 (X\ + x 2 ) 2 
D = b { x] + b 2 x\ 

If b\ = 0, the following equations result: 

( l/co“i ) V| - (2 {a/u n2 )y 2 + y, - ay 2 = (F/k x )/x m 
( 1 /<^ n 2 )$2 + ( 2 f / W n2 ) >'2 + >’2 = — ( 1 / <^*2 ). V | 
where jc m is some reference value of displacement and 

Vj = A' | fx m ; y 2 - A 2 /A m ; a = * 2 /*i 

= k\lm x ; u 2 n2 = k 2 /m 2 ; 2f = u n2 b 2 /k 2 

[Notes N9.2 and N9.3] 

NOTES FOR CHAPTER 9 

N9.2. Use of/„, as it has been defined in this chapter, implies that the system is oscillating in its fundamental 
mode. However, both Rayleigh’s and Lagrange’s methods are applicable to finding the frequency of higher 
modes of oscillation (harmonic modes); it is necessary merely to assume the shape of the oscillation or 
vibration mode and to compute the energies associated with that mode. 

N9.3. Optimum conditions for suppression of excessive vibration require a choice of = m 2 /m x and control 
of f to keep maximum vibration-amplitudes sufficiently small. If F = F m sin(2irft) and x m is chosen as 
F m /k y , then m 2 , b 2 , and k 2 should be chosen so that 


W n2 /«fll = ( 1 + 0) 1 

f 2 = (3/8)j3/(l + 0) . 

With these choices, the maximum amplitude x ] max of vibration of system 1 will be given by 

Ul.max/*m) 2 = 1 + 2/0 

and will occur at frequencies given by 

(/•//„,) 2 = (1 ±V[/3(2 + 0)]j/( 1 +0) . 

The maximum deflection x 2mBX of spring 2 will occur at these same frequencies and will be given by 

* 1 + (1/0) • 

An alternate formula for this maximum amplitude, when fdoes not have the optimum value but is near to it, is 

x 2 , m Jx m * (0.78/Vf)(l + 1/0) 3/4 . 

The value of 0 must be large enough so that the amplitude x 2max is acceptably small. 
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CHAPTER 10. MORE COMPLEX DYNAMICAL SYSTEMS. 

ANALOGIES 


10.0 Terminology . The symbols of Chapters 6 and 7 will also be used here. Other symbols are defined 
in Table 10.1 or will be defined as they are introduced. 

10.1 Analogies between electrical systems and mechanical or thermal systems . Table 10.1 summarizes 
results that have been derived in Chapters 6 and 7 concerning typical physical systems obeying the simple 
first- and second-order linear differential equations with constant coefficients. Not only do the equations 
for different systems become identical when expressed in terms of Y, y, T, 7, and f, but also there appear 
to exist analogous mathematical relationships for the various dependent physical variables. Thus, any pair 
of equations may be selected and an analogy drawn between the variables that appear in similar positions 
in the table. For example, (mass, linear displacement, force) and (moment of inertia, angular displacement, 
torque) constitute analogous groups of patterns. That these pairings or groupings are not unique is shown 
by the fact that each of the above groupings is also analogous to both (mass per unit area, particle displacement, 
pressure) and (mass/area 2 , volume displacement, pressure) for the pneumatic system. Insofar as mechanical 
and pneumatic systems are concerned, however, there exists a physical analogy as well as a mathematical 
analogy among the various factors, because all the relations are derivable by use of Newton’s second law 
of motion, and can be converted into each other by simple multiplication by some geometric factor. On 
the other hand, the analogy between mechanical and electrical systems is solely a mathematical one, rather 
than a physical one (unless one deals exclusively with power or energy). The analogy between thermal 
systems and electrical or mechanical systems is also solely of a mathematical nature. 

The analogies can be carried over into the treatment of physical systems of greater complexity than those 
treated in Chapter 6. The analogies may also be used for static problems, where time is not involved, as 
in problems of the potential (e.g., stress- or strain-contour lines in elasticity 1 , current- and voltage-contour 
lines in an electrically conducting medium, pressure- and velocity-contour lines of hydrodynamic flow, contour 
lines of heat-flow rate and of temperature in a thermally conducting medium). 

In experimentation on mechanical or thermal systems, it is often found that substantial improvements 
can be achieved in accuracy of measurement, simplicity of apparatus, flexibility of adjustment, or convenience 
of use, if an analogous electrical system is studied rather than the actual system of interest. This situation 
is so common that electrical terminology has been carried into other fields (e.g., “acoustic capacitance,” 
“acoustic resistance,” “thermal capacitance”) and electrical symbols have been used to denote mechanical 
quantities (e.g., acoustic inertance L a , mechanical compliance C m , thermal resistance R e ). The definitions 
of such nonelectrical terms are arbitrary and depend upon the particular pair of analogous equations chosen. 
(The acoustic terms defined in Tables 1-1 and 10. 1 are those most popular in the U.S.) Once the analogous 
pairings have been selected, any type of physical system, however complex, may be represented in analogy. 

For dynamical performance studies, it is necessary either that the combinations of physical quantities 
that have the dimensions of time have the same numerical magnitude in the actual and in the analogous 
system, or else that they be in constant proportion to each other. (For example, if the constants of an actual 
second-order system are T and r and the constants of the analogous second-order system are T and r\ 
then one may choose T = nT and r' = nr, in which case the “real” time t and the “analog” time t ' 
will be related by t' = nt.) 

Orthogonal quantities are tensile stress and shearing stress, or tensile strain and shearing strain. 
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In particular, two types of analogies between mechanical and electrical quantities have been used. In one 
of these analogies, the mechanical quantities ( m , b, k f x, x, F) are treated as analogous to the electrical 
quantities ( L , R, 1/C, q , q , £), respectively. This is the more common type of treatment. In another analog 
arrangement, which is often easier to employ in treating a complex mechanical arrangement, the quantities 
(m, b, k, x , x, F) are treated as analogous to the quantities (C, G, 1 /L, \edt, e, I), respectively. Figure 
10.1 lists some typical analogous arrangements. 
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l : igure 10. 1 -Mathematically analogous physical systems. 
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Figure 10. 1 .—Continued. 
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10.2 Complex systems. The equations of the two simple types of systems treated in detail in Chapter 7 
are describable mathematically as first- and second-order linear differential equations with constant 
coefficients. From the mathematical viewpoint, more complex systems may involve equations of the same 
order, but with variable coefficients or with nonlinear terms, or they may involve equations of higher order. 
From the physical viewpoint, these complex systems may involve situations where the parameters T , f, 
or r are themselves functions of time (a case of variable coefficients), where the damping is proportional 
to a higher power of the generalized velocity (Sec. 9.3) than the first power (a case of nonlinearity), or 
where there is more than one simple second-order system and the several systems are linked together (yielding 
a higher-order system). An extreme but common example of the last situation is the case where some or 
ail of the constants are distributed rather than lumped. 

In the case of first- or second-order systems with variable coefficients or nonlinear terms, it is sometimes 
adequate to treat the variable terms as though they were constant over a limited region of operation or to 
replace the nonlinear terms by linear ones over a limited region of operation, and thereby to utilize the 
results that have already been obtained in Chapter 7. Such approximations generally represent a considerable 
savings in effort over the treatment of the exact equations. 

In the case of higher-order systems which are linear, it is possible, as indicated in Sec. 7.4, to represent 
these as a tandem combination of component subsystems that are of the first or second order. Separate 
differential equations 

(10.2-1) MyMi) =o 

may be written for each of these subsystems. Each of the equations (10.2-1) has its own r or its own r 
and T n , depending on whether f represents a first- or second-order equation, respectively. Another algebraic 
equation or parametric set of equations may be needed to describe the relation among the dependent variables 

(10.2-2) F(y h y 2 ,..J f ,> •■) =0 

if such a relation is not already included in Eq. (10.2-1). The simultaneous solution of all of these equations 
will lead to an exact description of the entire system. 

An example of a system in which this procedure is needed is the dynamic vibration absorber 
(Example 9.3c) [Note N1 0.2]. On the other hand, mere inspection of the individual equations (10.2-1) may 
provide an intuitive perception of the system’s behavior. 

There are also many occasions where an adequate indication of system behavior can be obtained by use 
of approximation techniques that treat the complex system as though it were a simple first- or second-order 
system. Conditions that are sufficient to permit such approximations and to yield adequately accurate deduction 
of the forcing function Y(t) from the response y{t) are 

A. in a serial arrangement of subsystems represented by Eq. (10.2-1), the energy used by any subsystem 
is only a small fraction of the energy available from the preceding subsystem; 

B. sufficient time has elapsed for the decay of the transients that occurred on the initiation of K(/); 

C. the value oj max of the highest angular frequency in Y(t) that is of interest is small compared to the 
values of 1/r and of the subsystems. This condition may be excessively conservative (although 
sufficient, it may not be necessary), as will be indicated in Example 10.5a. 

In this chapter, some examples of approximation techniques will be presented. 
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10.3 First-order system with variable time constant . If r is not constant, but depends on either t or y so that 

(10.3-1) 7 = r(/) or t = r(y) 

then the forcing function Y(t) may be deduced from the response y(f) by piecewise application of the 
procedure outlined in Sec. 7.13. Each point ( /,>’) is replaced by (t,Y), where 

(10.3-2) Y = y + T(dyldt) 

and r is given by Eq. (10.3-1). The practical limitation of this procedure is usually determined by the accuracy 
of determining dy/dt . 

An approximation to this procedure is possible if A^ n , as defined in Sec. 7.13, is much larger than the 
largest value of t. Then, each point (t,y) may be replaced by (/ -I- r, y). The resultant curve is Y(t). 

10.4 Second-order system with square-law damping . In some instruments such as air-damped 
accelerometers, iron-vane-type ammeters, and pneumatic systems containing orifices, a considerable part 
of the damping force is proportional to the square of the velocity rather than to the first power of the velocity. 
For such and similar systems, the equation of dynamic response may be written as 

(10.4-1) y/(4ir 2 /^) + b 2 |y|y + brf + y = Y 

in place of Eq. (7.1-2a). 

The behavior of this system can be understood by writing the response equation as 

(10.4-2a) y/(47 x 2 f 2 n ) + ry + y = Y 

where 

(10.4-2b) T = fc 2 |y| + f>, = r /(*/„). 

The system resembles that of Sec. 7.1, except that the time constant t and the fraction of critical damping 
f vary with the velocity amplitude |y|. At sufficiently large values of |y|, where f > > 1, the system may 
appear overdamped; at sufficiently small values of |y |, if b\ < < \/(rf n ), the system may appear to have 
negligible damping. 

A system with predominantly square-law damping that appears to be well damped for large velocity 
amplitudes of the forcing function will appear almost undamped for very small velocity amplitudes of the 
forcing function. Only the first-power (linear) damping that invariably remains in any real physical system 
then serves to continue to dissipate energy. 

The comparative response of a linear system and a square-law system (when b\ < < b 2 |y|) to a step 
change that produces the same first overshoot is shown in Fig. 10.4. In response to a step change of magnitude 



(a) <b) 

(a) System with linear damping. 

(b) System with square-law damping. 
Figure 10.4 — Response to a step change. 
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( - Y 0 ), from an initial value of Y 0 , the approach to the steady-state value of 0 will be through a damped 
oscillation. The ratio between successive amplitudes of the oscillation will not be constant, as in the basic 
linear system, but will approach unity as the oscillation continues. Thus, the oscillation will be attenuated 
less and less as the oscillation continues. The approach to the final value would never become aperiodic, 
no matter how great b 2 may be, if b ] were zero. [Note N10.4] 

If the system is overdamped because b, in Eq. (10.4-2b) is appreciable, then the system may be 
considered to resemble a first-order system with slightly variable 7 , and the techniques of Sec. 10.3 may 
be applicable. 


Ri 

o — vw 


e 0 “ Yu > 

o— 
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y(t) 


Figure 10.5 . 1 — ' Two First-order systems in tandem. 


10.5 Examples of higher-order systems in which superposition is possible . 

Example 10.5a. The circuit of Fig. 10.5.1 may represent 

(1) an electrical circuit consisting of two first-order systems connected in tandem, under the 
restriction that C 2 < < C] 

(2) the analog of a resistance-thermometer winding centered in a protective well that is immersed 
in a flowing fluid whose temperature is to be measured. The analogous quantities are 

e 0 the temperature of the fluid 

R ] the thermal resistance of the fluid in contact with the well 

Cj the thermal capacitance of the well 
e t the temperature of the well 

R 2 the thermal resistance of the material separating the resistance-thermometer 
winding from its protective well 

C 2 the thermal capacitance of the resistance-thermometer winding 
e 2 the temperature of the winding 

These quantities are defined more precisely in Table 10.1 and Sec. 6.12. 

If condition A of Sec. 10.2 exists, the output e x (r) of the first system may be determined from 
the input K(r), the time constant rj, and the information in Secs. 7.7 to 7.13, neglecting the presence 
of the second system. This output may then be treated as the input to the second system alone. 

The tandem combination of the two systems acts like an overdamped second-order system with 
time constant t ] + t 2 . 


Example 10 ,5b. The circuit of Fig. 10.5.2 may represent 

(1) an electrical circuit consisting of a first-order system connected to a second-order system 
(Sec. 6.8), under the restriction that C 2 < < C, 

(2) the analog of a thermocouple measuring fluid temperature, as in Sec. 6.12, generating an 
electrical output that is measured by an electromagnetic oscillograph, as in Sec. 6.3. The 


O-AVv — t |— 

e 0 = Y(t) Cj^p e | C 2 i 

o ' — c4q L_ 


e 2 = y<t) 
O 


Figure 10.5.2— A first-order and a second-order system 
in tandem. 
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analogous quantities are 
€q the temperature of the fluid 

the thermal resistance of the fluid in contact with the thermocouple 
C x the heat capacity of the thermocouple 

e x in turn, 

(i) the temperature of the thermocouple 

(ii) the electric current delivered to the oscillograph because of the emf generated 
by the thermocouple 

R 2 the electromagnetic damping torque of the oscillograph produced by eddy currents 
and Lenz law currents 

L 2 the rotational inertia of the oscillograph 
C 2 the spring stiffness of the oscillograph 
e 2 the actual oscillograph deflection 

These quantities are defined more precisely in Table 10.1 and Secs. 6.3 and 6.12. 

(3) the analog of a pressure gauge, as in Sec. 6.1, being used to measure the pressure in the 
chamber of a pneumatic system, as in Sec. 6.6, whose capillary is so small or so long that the 
pneumatic system is overdamped. The analogous quantities are 
e 0 pressure at the entrance of the capillary 
R x the pneumatic resistance of the capillary 
Cj the compressibility of the gas in the chamber 
€ x the pressure in the chamber 

R 2 the mechanical damping of the pressure gauge (often negligible) 

L 2 the mechanical inertia of the pressure gauge 
C 2 the spring stiffness of the pressure gauge 
e 2 the pressure indicated by the gauge 

These quantities are defined more precisely in Table 10.1 and Secs. 6.6 and 6.7. 

The pressure gauge volume may be included as part of the chamber volume entering 
into Cj. If a correction ay p 0 for the change in volume V (Case E of Sec. 6.7) is needed, 
the condition 


cty. Po << y 


should be met. 

The output of the first-order system may be determined from the input Y(t), the time constant 
7 ,, and the information in Secs. 7.7 to 7.13, neglecting the presence of the second-order system. 
This output may then be treated as the input to the second-order system alone, using the information 
in Secs. 7.15 to 7.21. 

If 7 2 < < 7 1 and T n2 < < 7, then e 2 {t) closely resembles e x (t). 

If f 2 > > 1 , then 


e 2 (t) * Y(t - 7 , - 7 2 ). 

Example 10.5c. The diagram of Fig. 10.5.3 may be considered to represent 
(1) an electrical circuit consisting of two second-order systems, connected in tandem, with 
C 2 << C,; 


R, L| ’ 1 R 2 L-> 1 

6-AV^TJOtPn 6-J-6 — VVWTJOC^-i o 

eo = Y(t) C|i Cj Cii e>sy(t) 

0 1 9-^9 1 9 

1 i I 1 


Figure 10.5.3— Two second-order sysiems in tandem. 
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(2) a pressure gauge, as in Sec. 6.1, connected to the chamber of a pneumatic system, as in 
Secs. 6.6 and 6.7. The analogous quantities are the same as in Example 10.5b(3) and, in 
addition, 

Lj the mass of gas in the pneumatic tube. 

If C 2 < < Cj and if Y(t) contains no sinusoidal component of appreciable amplitude that has a 
frequency comparable to f n2% the output of the first system may be determined from Y(t), the 
parameters T] and f nl , and the information in Secs. 7.15 to 7.21, neglecting the presence of the 
second system. This output may then be treated as the input to the second system alone. 

If r 2 < < Tj and T n2 << T n] , then e 2 (t) closely resembles e\ (f). Fidelity in achieving the 
condition 


e 2 U) - Y(t) 

is always more likely if ft and ft are close to 0.7. 

If ft > > 1 , ft > > 1 , then 


e 2 (t) *= Y(t — T] — 72 ). 


Example 10.5rf. Figure 10.5.4(a) shows a pneumatic system consisting of an arbitrary arrangement 
of capillary tubes and volumes. Its approximation depends on whether the imposed pressure changes 
are sufficiently slow so that fluid inertia effects can be neglected. For any one of the tubes, a reasonable 
criterion is that /shall be less than tv/A, where 

/ frequency of the imposed pressure fluctuations 
v kinematic viscosity of the gas 
A cross-sectional area of the tube 


tZt 




(a) 


R 1 R 3 R4 R6 

I | 1 

L ciTcfT cfT otT csT diT ctT j 


(b) 


L6 


R] R3 L4 
j VjV— '"fljKP — j— 

L CiTciT C3f C4T C5TC6T ctT 

(c) 


(a) Pneumatic system. 

(b) “Series” electrical analog when inertial effects 
are absent. 

(c) “Series” electrical analog when inertial effects 
are present. 

Figure 10.5.4— Several systems in tandem. 


A. If this criterion is satisfied for every one of the tubes, the system consists solely of acoustic 
resistances and capacitances, and is analogous to the circuit of Fig 10.5.4(b). The entire system 
can be treated as a first-order one with an equivalent time constant that is a function of the separate 
time constants of various portions of the system. The data of Table 10.1 and Sec. 6.7 may be used 
to compute these time constants. For the basic combination of a tube and a chamber, the time constant 
may be written as 


t = BC(V+AI/ 2) 


where 


B = 8 p/(ira 2 ); C — z 2 l/A 2 . 
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Here B is a property of the gas, C is a property of tube geometry, V is chamber volume, and Al/2 
represents one half of the tube volume. 

For the specific circuit of Fig. 10.5.4(b), the equivalent time constant of the entire system, 
representing the response e to the forcing function E is 



Each summation of capacitances in this formula is analogous to the summation of all volumes 
downstream of the respective tube. 

B. If the criterion {f < irv /A) is not satisfied for some particular tube, the inertia of the gas 
in the tube may become significant. For example, if the criterion is not satisfied for tubes 4 and 
6 in Fig. 10.5.4(a), the electrical analogy becomes Fig. 10.5.4(c). Usually, this system may be treated 
conveniently as the superposition of one first-order system and two second-order systems only when 
criteria A, B, C of Sec. 10.2 are met. If they are not met, and the subsystems are underdamped, 
the system’s response to a step or impulse may show oscillations at each of the resonant frequencies 
of the subsystems and at the sums and differences of those frequencies. Conversely, if the transient 
response to a step or impulse does show such oscillations, it is unlikely that superposition techniques 
will be useful in estimating the system’s behavior. Solution using Eqs. (10.2-1) and (10.2-2) may 
be necessary, unless the system is of the types treated in Sec. 10.6 [Note N10.5]. 

10.6 Systems with uniformly distributed parameters. When parameters like resistance, inductance, and 
capacitance, or their analogs, are uniformly distributed along a length / of considerable magnitude, the system’s 
behavior often resembles that of a first-order or second-order system. Four examples will be given here. 

(1) A line of considerable length that behaves predominantly like a first-order system 

(2) A line of considerable length that is terminated by a “load” in such manner that the system 
behaves like a first-order system 

(3) A line of considerable length that behaves in most respects like a second-order system 

(4) A line of considerable length that is terminated by a “load” in such manner that the system 
behaves like a second-order system 

Example 10.6 a. The problem of temperature change of one end of a long bar of constant cross 
section (Fig. 10.6. 1(a)) whose other end is subjected to a fluctuating temperature 0, where the bar 
has length /, cross-sectional area A, density p, thermal capacity (specific heat) c, and thermal 
conductivity k, may be approached by dividing the length of the bar into elements of length Ax, 
each of which is considered as a first-order system similar to that of Sec. 6.12. Each element has 
mass pA*Ax, thermal capacity c , and thermal resistance Ax/(kA). 

The analogous “series” electrical circuit may be represented by two parallel conducting wires, 
each of length / and resistance R/ 2, such that the capacitance between them is C. This system may 
be represented by a sequence of first-order systems whose elements are the resistance (R/l)*Ax 
and the capacitance (C/l) 9 Ax. This system is shown in Fig. 10.6.1(b). 

The analogous “parallel” electrical system may be represented by Fig. 10.6.1(c), wherein the 
elements are (L/l)*Ax and (G//)*Ax. 

An analogous pneumatic system is a section of a long gas-filled tube of uniform cross section. 
If inertia forces are negligible compared to viscous forces, any element of length Ax has an acoustic 
resistance (Sxr}/A 2 )*Ax and an acoustic capacitance A 9 Ax/(yp 0 ). The equivalent model is shown 
in Fig. 10.6.1(d). 

The output (temperature in Fig. 10.6. 1(a), open-circuit emf in Fig. 10.6. 1(b), short-circuit current 
in Fig. 10.6.1(c), and pressure in Fig. 10.6.1(d)), measured at one end of the system, in response 
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Revised terminations 
for Example I0.6.b. 



(a) Thermal system. 

(b) “Series” electrical system. 

(c) “ Parallel’ ’ electrical system. 

(d) Pneumatic system. 

Figure 10.6. 1— Long, uniform line represented as a first-order 
system. 


to an input at the other end, resembles that of a first-order system (Secs. 7.7 to 7. 13) having a time 
constant r equal to 


l 2 /( 2 k) 

RC/2 

(10.6-1) LG/2 

4tv1 2 / (a 2 A) 


for the thermal system ( k « diffusivity) 
for the ‘‘series” electrical system 
for the “parallel” electrical system 
for the pneumatic system [Note N 10.6.1] 


provided that condition C of Sec. 10.2 is met. If it is not, then the system behaves like an overdamped 
second-order system (Secs. 7.15 to 7.21), with r as given by (10.6-1). 


Example 10.6 b. If Example 10.6a were modified by adding a termination at the output end as 
shown at the right-hand side of Fig. 10.6. 1 (a larger thermal mass, a larger capacitor, a larger inductor, 
or a larger volume), then the systems would behave as though their time constants were 
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for the thermal system 


±(eHs + M c 

kA\ 2 


(10.6-2) R{ - + C, 


G {~2 + L ' 


8 * vl(M + v 


2 <A: 


for the “ series* ’ electrical system 
for the “parallel’ ’ electrical system 
for the pneumatic system 


The behavior would be similar to that described for Example 10.6 a, except that, as any termination 
became a larger fraction of the quantity shown in parentheses, the system’s behavior would approach 
more nearly that of a basic first-order system. 

Example 10.6c. If, in the electrical and pneumatic systems like those of Example 10.6a, the 
distributed inertance of the pneumatic system, the distributed inductance of the series electrical system, 
and the distributed capacitance of the parallel electrical system cannot be neglected, the models of 
these systems must be replaced by those of Fig. 10.6.2. Each of these systems possesses the following 
features: 

(1) It is capable of oscillation. In particular, it is also capable of oscillation at frequencies that 
are harmonics of its fundamental mode. 



8 ,.* ^ 

4 *T- # AX 



Revised terminations 
for Example 10. 6d 


(a) “ Series'’ electrical system. 

(b) “Parallel** electrical system. 

(c) Pneumatic system. 

Figure 10.6. 2. —Long, uniform line represented as a second- 
order system. 





§10. 6-Notes 


(2) In its fundamental mode, it behaves like a second-order system having a natural period T n 
and time constant r given in Table 10.6c: 


Table 10.6c 


System T 2 

J n 

Pneumatic tube [Note N 10.6.2] 16 I 2 /a 2 

“Series” electrical line 16 LC 

“Parallel” electrical line 16 LC 


T 

4wvl 2 / {Aar ) 

RC/2 

LG/2 


(3) In response to a suddenly applied input, no response will be obtained at the output until 
a delay time t 0 = T n { 4 has elapsed. Thereafter, the system’s response will resemble that 
of the second-order system whose natural period and time constant are listed above. 

Example 10.6 d. If Example 10.6c were modified by adding a termination at the output end as 
shown at the right-hand side of Fig. 10.6.2 (a larger volume, capacitor, or inductor), then the systems 
would retain the characteristics listed for Example 10.6c except that 

(a) the natural periods and time constants listed in Table 10.6c would be multiplied by the factors 
listed in Table 10. 6d 

(b) the delay time r 0 would still be approximately equal to T n /A, where T n is listed in 
Table 10.6c. 


Table 10. 6d— Multipliers for T 2 n and t Listed in Table 10. 6c. 


System 


Multiplier Multiplier 

for T\ for t 


Pneumatic [Note N 10.6.2] l + xt7(4/r) 

“Series” electrical 1 + t 2 C,/(4C) 

“Parallel” electrical 1 + x 2 L ] /(4L) 


1 + 2K/(*r 2 /) 
1 4- 2C,/C 
1 + 21, /L 


NOTES FOR CHAPTER 10 

N10.2 The electrical analogues of Fig. 9.3 are shown in Fig. 10.2. A convenient method of obtaining the 
steady-state solutions for such circuits is indicated in Note N10.5. The differential equations of Example 
9.3c are derivable in such manner. 



l L, 

(a) 1 Cb) 


(a) “Series” circuit analog. 

(b) “Parallel” circuit analog. 

Figure 10.2.— Electrical analogs of the dynamic vibration * 
absorber (Fig. 9.3). 
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§Ch. 10-Notes 

N10.4 In the extreme case where b j is negligible, the system is characterized by two constants, like b 2 
and f n , and a third constant that has the dimensions of y. Let 

0 = 47 rfnWo • 

If y n is the ordinate of the nth overshoot (n = 0,1,2,... ), and ordinates with n even are of opposite sign 
to ordinates with n odd, then the successive ordinates are given by 

(1 - 20z„ +1 )exp(20z„ +I ) = (1 + 20z„)exp(-20z„) 

where z n = \y n \/Y 0 

and Zo = 1 

An empirical approximation is 

Z n — ( 1 + 1.4 n0)- 1 

except for the case (n = 1, 0 > 1). 

For the first overshoot ( n = 1), 

*,-1-40(1-0/3 if 0 < 0.3 

z, * 1/(20) if 0 > 3 

The time interval between the initiation of the step and the first crossing of the asymptote is approximately 

(1 + /3/4)/(4/„) if 0<1O 

0.2/3 2/3 //„ if 0 > 10 . 

Succeeding oscillations occur at a frequency approximately equal to /„. 

N10.5 A convenient way to determine the steady-state response of any of the electrical analogs shown in 
this chapter, to a sinusoidal forcing function of angular frequency w, is to represent the impedances of 
inductances and capacitances as jwL and l/(/«C), respectively (where j = V(-l)) and then to determine 
the ratio (output/input) for the particular circuit. Thereby, the problem becomes an algebraic one, rather 
than one of differential equations. 

N10.6.1 This result is valid when Poiseuille-type flow exists; namely, that, if a pressure fluctuation 

P = P m sin (wr) 

occurs at the inlet end of the tube, the condition 

Au/v < 2 T 2 

is then satisfied. Here, A is the cross-sectional area of the capillary tube and v is the kinematic viscosity 
of the fluid. 
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§Ch. 10-Notes 


N10.6.2 The pneumatic time constant given in Table 10.6c is correct only when there is Poiseuille-type 
flow. The criterion for this condition is given in Note N10.6. 1 . When Poiseuille-type flow does not occur, 
inertial forces become dominant over viscous forces, the system is almost undamped, and r may be neglected. 

The resultant natural vibrations (“organ-pipe vibrations 1 ’) may obscure a measurement, made at the outlet 
end of the tube, that is intended to measure the instantaneous value of a fluctuating inlet pressure. An expedient 
used to provide a reliable measurement is to make the tube of substantially infinite length and to measure 
the pressure at some convenient location at a relatively short distance x from the inlet end. The installation 
of a pressure gauge at this location must be made in such manner that 

(a) there is no significant change in the cross-sectional area of the gas flow path; 

(b) there are no protrusions in the gas flow path; 

(c) any cavity between the gas flow path and the pressure gauge is of negligible acoustic capacitance 
or inertance. 

If a pressure fluctuation 


P = P m sin (a )t) 

occurs at the inlet end of a circular tube of radius r, the pressure at a distance x from the inlet end will 
be attenuated so that the amplitude is 


where 


a = 0 (ar) ~ 1 V(o>y) 

provided the tube is of substantially infinite length, so that reflections from the output end will be so weak 
that organ-pipe oscillations will not interfere appreciably with the measurement. When Au is so small that 
Poiseuille-type flow exists (/4a> < 2 k 2 p), the proportionality constant 0 is equal to 2. At larger values of 
Aii), where inertial forces become prominent, the proportionality constant 0 , for air, is near to unity. 

“Substantially infinite length” implies a length / such that e _or/ < < 1. For example, if a/ > 3, the 
attenuation at the outlet end will be more than 20-fold. 

This technique of pressure measurement is termed the “infinite-line technique.” In acoustics, it forms 
the basis of some designs of probe microphones. 

In the analogous electrical or thermal systems, the infinite line technique remains applicable. The electrical 
analogies have often been used, in either experimental or mathematical form, to analyze pneumatic situations. 
In these analogs, the attenuation constant a is given by 

a = V (t ot/1 2 ) 


where t is given in (10.6-1). 
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TABLE 1-1. -PHYSICAL QUANTITIES IN MECHANICS 


Symbol, 

definition 

Name 

System of dimensions 


M,L,T 

F,L,T 

/ 

Length 

L 

L 

t 

Time 

T 

T 

m 

Mass 

M 

FL-'T 2 

F 

Force 

MLT -2 

F 

4> 

Angle 

1 

1 

0 

Solid angle 

1 

1 

A 

Area 

L2 

L2 

V 

Volume 

L3 

L* 

V 

Velocity, linear 

LT-» 

LT~i 

c a 

Speed of sound 

LT-* 

LT-> 

<t> 

Velocity, angular 

T-> 

T-i 

V 

Velocity, volume 

L 3 T _I 

L 3 T-' 

m 

Velocity, mass 

MT-« 

FL-'T 

a = v 

Acceleration, linear 

LT~ 2 

LT~ 2 

g 

Acceleration of gravity, local 

LT~ 2 

LT- 2 

gn 

Acceleration of gravity, standard 

LT-2 

LT- 2 

a = 0 

Acceleration, angular 

T-2 

T-2 

G 0 = Fl 2 /(m l m 2 ) 

Gravitational constant 

M-'L 3 T-2 

F-'L 4 T- 4 

h = 

Moment of area 

L 4 

L 4 

4. = ™' 2 

Moment of inertia 

ML 2 

FLT- 2 

p = m/K 

Density 

ML ~ 3 

FL- 4 T 2 

1/P 

Specific volume 

M-'L 3 

F-‘L 4 T-2 

II 

? 

Specific weight 

ML" 2 T- 2 

FL-3 


Weight 

MLT -2 

F 

P m = "IV 

Momentum, linear 

MLT-' 

FT 

M = Ft 

Moment of a force 

ML 2 T -2 

FL 

K = U 

Moment of momentum 
(angular momentum) 

ML^" 1 

FLT 

/ = F/ 

Impulse 

MLT" 1 

FT 

j = Wr 

Action 

ML 2 T"' 

FLT 

p ~ F/A 

Pressure 

ML-'T- 2 

FL-2 

P = Jf'/r 

Power (Flux) 

ML 2 T- 3 

FLT- 1 

P//4 

Power density (flux density) 

ML" 3 

FL - , T“ 1 

AP/Af 

Power spectral density 

ML 2 T -2 

FL 

W — FI 

Energy 

ML^" 2 

FL 

W/V 

Energy density 

ML-'T- 2 

FL-2 

M/= f|/^2 

Coefficient of friction, sliding 

1 

1 

#<» = /Ei/Ei 

Coefficient of friction, rolling 

L 

L 

r = f/i 

Surface tension coefficient 

MT- 2 

FL~‘ 

V = F//(Av) 

Viscosity 

ML-'T-' 

FL-2T 

1/1 

Fluidity 

M-'LT 

f- 1 l 2 t- 1 

* = >?/p 

Kinematic viscosity 

L 2 T-' 

l 2 t _1 

D = m/M • Ap/A/) 

Diffusion coefficient | 

L 2 T-' 

L2T-« 




TABLE 1-1. -Concluded. 


Symbol, 

definition 

Name 

System of dimensions 


M,L,T 

F,L,T 

* tih 

Product of inertia of an area 

L 4 

L 4 


Product of inertia of a body 

ML 2 

FL” 1 

z = i A n 

Section modulus 

V 

V 

c „ = u/i 

Strain, normal (longitudinal) 

1 

1 

y = A<t> 

Strain, shear 

1 

1 

a„ = Ft A 

Stress, normal 

ML-'T- 2 

FL’ 2 

11 

>1 

Stress, shear 

ML -1 T~ 2 

FL- 2 

& = (ak/h/Ap 

Compressibility 

M-'LT 2 

F-'L 2 

1/0 

Bulk modulus 

ML-'T- 2 

FL“ 2 

= ° n / f n 

Modulus of tension (modulus of 
elasticity, Young’s modulus) 

ML-'T-2 

FL -2 

G = T/y 

Modulus of shear 

ML-'T- 2 

FL 2 

C a = V8 oc. AV/Ap 

Acoustic compliance 

M~'L 4 T 2 

F-'L 5 

R a = Ap/AV 

Acoustic resistance 

ML- 4 T-' 

FL _5 T 

L a = m/^ 2 

Acoustic inertance 

ML- 4 

FL- 5 T 2 

P 

Acoustic source strength 

L3T-' 

L 5 T-' 

^c a /Af 

Acoustic dispersion 

L 2 T-' 

L 2 T-' 

C s = Ap/Av = pc Q 

Specific acoustic resistance 

ML“ 2 T~' 

FL-’T 


TABLE 1-2 -PHYSICAL QUANTITIES IN HEAT 


Symbol, 

definition 

Name 

Dimensions 


MLT6 

WLT0 

0 

Temperature 

0 

0 

m 

Mass 

M 

WL~ 2 T 2 

p = m/V 

Density 

ML- 2 

WL _5 T 2 

W 

Energy 

ML 2 T~ 2 

W 

P 

Power 

ML 2 T~ 3 

WT-' 

Q = AW/ AS 

Thermal capacity 

ML 2 T- 2 0-' 

W0-' 

c = AW/ {m • AS) 

Specific heat 

L 2 T- 2 0-' 

L 2T- 2 0-' 

AW/{V*AB) 

Thermal capacity 
per unit volume 

ML-'T- 2 0-' 

WL -3 0 - 1 

AW/ Am 

Latent heat 

L 2 T- 2 

L 2 T“ 2 

H = W/m 

Enthalpy 

L 2 T- 2 

L 2 T~ 2 

AS = AW/6 

Entropy 

ML 2 T- 2 9-' 

W0-' 

AS/m 

Specific entropy 

L 2 T- 2 6-' 

L 2 T- 2 0-' 

h = P/ {A* AB) 

Heat transfer coefficient 

MT-50-' 

wl- 2 t-'0-‘ 

* x = P/{A • AB/Al) 

Thermal conductivity 

MLT- 3 0-' 

WL-'T-'0-' 

* = k x /{pc) 

Thermal diffusivity 

L 2 T-' 

L^T -1 

a L = Al/{1* AB) 

Coefficient of expansion, 
linear 

0-' 

0-' 

a y = AVf ( V • A0) 

Coefficient of expansion, 
cubical 

0-' 

0-' 

£ 

II 

Boltzmann constant 

ML 2 T- 2 0-' 

w©-' 

Rq = W/(md) 

Universal gas constant 

L 2 T - 2 0 — ' 

L 2 T- 2 0-' 


"Applicable laws: &W * &{pV) * 4(05) 





TABLE 1-3.— ANALOGOUS QUANTITIES IN ELECTRICITY AND MAGNETISM 


Electricity 

Magnetism 

Symbol, 

Name 

Dimensions, 

Name 

Symbol, 

definition 


RILT system 


definition 

/ 

Current 

I 




Q = It 

Charge 

IT 

RIT 

Magnetic pole strength 

m* 

* = «? 

Electric flux 

IT 

RIT 

Magnetic flux 

4> = 

M„ = Ql 

Electric moment 

ILT 

RILT 

Magnetic moment 

M m = m*l 

D = */A 

Electric displacement 

IL _2 T 

RIL _J T 

Magnetic induction 

B = 4>/A 

= e 0 £ + 8P 




(magnetic flux density) 

= *>H + 8J 

P q = M q fV 

Electric polarization 

IL~ 2 T 

RIL- 2 T 

Magnetic polarization 

J = MJV 

= «ox,e 




(magnetization, intensity 

= MoX Ji 





of magnetization) 


£ = AR'/O = « 

Electromotive force 

RJ 

I 

Magnetomotive force 

T = AW/m' 


(electric potential 



(magnetic potential 



difference) 



difference) 


£ = £// = F/£? 

Electric field strength 

RIL 1 

IL- 1 

Magnetic field strength 

H = $/l = F/m * 


(electric field intensity, 



(magnetic Held intensity. 



electric potential 



magnetizing force) 



gradient) 





< = D/S 

Permittivity (absolute 

R-'L-'T 

RL _1 T 

Permeability 

H = B/H 


dielectric constant) 





e o 

Permittivity of a vacuum 

R^L-’T 

RL-'T 

Permeability of a vacuum 

Mo 

c r = </f 0 

Relative permittivity 

1 

1 

Relative permeability 

Mr * M/Mo 


(dielectric constant, 






specific inductive 






capacity) 





X V .V = («r-D//3 

Electric susceptibility 

1 

1 

Magnetic susceptibility 

Xm.v ~ (Mr" D/^ 


per unit volume 



per unit volume 


Applicable laws 

F =0Q\Qi/ (4«/2) 

Coulomb’s law 


Coulomb’s law F — /3m]mJ/( 4rpd 2 ) 


S = -0(d*/dt)/l 

Faraday’s law 


Faraday's 

law E = L(dl/dt) 





TABLE 1-4. -ADDITIONAL PHYSICAL QUANTITIES 
IN ELECTRICITY AND MAGNETISM 


Symbol, 

definition 

Name 

Dimensions, 
RJLT system 

t 

Time 

T 

l 

Length 

L 

A 

Area 

L2 

V 

Volume 

V 

m 

Mass 

R1 2 L~ 2 T 3 

F 

Force 

RI 2 L-'T 

r 

Power 

I 2 R 

w 

Energy 

I 2 RT 

R 

Resistance 

R 

g„ = UR 

Conductance 

R- 1 

o q = RA/l 

Resistivity 

RL 

P„-' = l/(AR) 

Conductivity 

R-'L-' 

C= Q/E 

Capacitance 

R-'T 

L 

Inductance 

RT 

« = 5/*™ 

Reluctance 

R-'T-' 

A = 1/(R 

Permeance 

RT 

a, = M g /U 0 &) 

Polarizability 

L 3 

d mn = Q/F 

Piezoelectric coefficient 

R-'I-'L"' 

S e = A E/A6 

Seebeck coefficient 

RI6 


(Thermoelectric power) 


Hi = AE/AO 

Thomson coefficient 

Rie-' 

n = p/i 

Peltier coefficient 

RI 

R h = A E/(BI/l) 

Hall coefficient 

I-'L 3 T-‘ 

p E = a e/(Bi/i) 

Ettinghausen coefficient 

R-'|- 2 L 3 T-'0 

Q n = AE/ {B • A$) 

Nemst coefficient 

L 2 T-'0-' 

Sl = {d$/db)/(B • dd/dl) 

Righi-Leduc coefficient 

R-'I-'L 2 ^' 







TABLE 1-5. -SOME QUANTITIES IN RADIATION AND ILLUMINATION 


Radiation 

Dimen* 

Illumination 



sions, 



Symbol, 

Name 

PLT 

Name 

Symbol, 

definition 


system 


definition 

Or 

Radiant energy 

PT 

Luminous energy 

Gv 

% = Q e /v 

Radiant energy density 

PTL-3 

Luminous energy density 

w v = QJV 

*< = Qe 

Radiant flux 

P 

Luminous flux 

*v =Gv 

I, = +,/« 

Radiant intensity of source 

P 

Luminous intensity of source 

/ v - <t> v /n 

M, = 

Radiant exitance of source 

M 

1 

eu 

Luminous exitance of source 

Af v = 4> v /d 


(radiant flux density) 


(luminous flux density) 


E, = *,//! 

Irradiance on receiver 

PL-2 

Illuminance on receiver 

£ v = * y /A 

l, = 

Radiance of source or 

PL-2 

Luminance of source or 

Ly = * V /(^Q) 


through cross section of path 


through cross section of path 
(was brightness) 




1 

Luminous efficacy (luminosity) 

a: v = 



1 

Luminous efficiency 
(relative luminosity) 

= Kv.x/Ky.max 



1 

Maximum luminous efficacy 

J Y 

M ^y.max 


Absorption coefficient 

L“ l 



iL'ip 

Mass absorption coefficient 

L«p-it-3 



n = C w /f 

Refractive index 

1 



2(n t -n 2 ) 

Dispersive power 

1 



(n,+n 2 - 2) 



<p/l 

Rotatory power 

L-> 



a = y?/(fp) 

Specific rotatory power 
(Specific rotation) 

L 4p-ir-J 




Common quantities 

Symbol 

Definition 

Dimensions 

/ 

Length 

L 

A 

Area 

L2 

V 

Volume 

L3 

*P 

Angle 

1 

0 

Solid angle 

1 

c 

Speed of light 

LT-' 

m 

Mass 

PL3T-3 

P 

Density 

PL-'T-J 





TABLE 1-5. -Concluded. 


Subscripts: 

e physical variable, related to radiation (sometimes omitted if illumination is not being considered) 

v psychophysical variable, related to illumination 

v,max the value at peak of standard human visibility curve 

X (may be a second subscript) the value, at wavelength X, of the derivative of the quantity with respect to X. (The 

derivative has different dimensions.) 

Suffixes: 

i) Terms with the ending “ivity” denote the intrinsic properties of material with a clean, polished surface. 

Terms with the ending “ance” denote the extrinsic properties of an actual specimen of a specified shape and 
surface condition. 

Some dimensionless quantities: For any specified type of radiation- 

p (reflectance) or p ' (reflectivity) = 

€ (emittance) or t’ (emissivity) = $ ( /$bb 

a (absorptance) or a ' (absorptivity) = $> a /$ ( = 

t (transmittance) = $ r /$. 

R (reflectance factor) = 

where 

= incident flux = absorbed flux = transmitted flux 

<7 = reflected flux = emitted flux 

^BB = ^ ux that would be emitted or absorbed by a blackbody under the same conditions* 

= flux that would be reflected by a perfect diffuser under the same conditions* 

“Same conditions" implies same specimen temperature, same radiation wavelengths, and same geometric conditions (such 
as specimen area, angles of incidence and view, and solid angles of the incident and reflected beams). 

Applicable laws: 

P\ + Q \ + = 1 

ot\ * = 1 - Px for an opaque body 

Note: For more complete information, see Dictionary of Lighting Terminology, Illuminating Engineering Society (IES) 
Lighting Handbook, 1981. 
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TABLE 1-6 -SOME DIMENSIONLESS NUMBERS 


Name 


Subscript 

Definition 

Name 

Subscript 

Definition 



of N 



of N 


Biot 


Bi 

hl/k 

Mach 

M 

v/c 0 

Clausius 


Cl 

v 3 /p/ {k • A6) 

Nusselt 

Nu 

hl/k 

Eotvos 


Eo 

pthtv 

Peclet 

Pe 

vI/k 

Euler 


Eu 

P/(PV 2) 

Poisson 

Po 

(E n /2G) - 1 

Fanning* 


Fa 

r/(pv 2 /2) 

Prandtl 

Pr 

ci)/k 

Fourier 


Fo 

*/(w/ 2 ) 

Rayleigh 

Ra 

#8/ 3 • A 0/{vk) 

Froude b 


Ft 

v 2 /g/ 

Reynolds 

Re 

\l/v 

Graetz 


Gz 

rhc/(kl) 

Schmidt 

Sc 

vfD 

Grashof 


Gr 

g&P • AO/v 2 

Stanton 

Sr 

h! (pev) 

Knudsen 


Kn 

\/l 

Stokes 

Sk 

\¥/{gl 2 ) 

Lewis 


Le 

D/k 

Strouhal 

Sr 

«//v 

Symbols 

c 

specific heat at constant pressure 

V 

velocity 


'a 

speed of sound 


& 

coefficient of cubical expansion 

D 

diffusion coefficient 

r 

surface tension coefficient 


modulus of tension 

V 

viscosity 


G 

modulus of shear 


AB 

temperature difference 

8 

local acceleration of gravity 

K 

thermal diffiisivity 

h 

heat transfer coefficient 

A 

mean free path of molecules 

k 

thermal conductivity 

¥ 

kinematic viscosity 

l 

characteristic length 

P 

density 


m 

mass flow rate 


r 

shear stress 


P 

pressure 


a) 

angular frequency 


a Fanmng factor / = where r ^ • hydraulic radius, / * length over which bp occurs, bp * t x (l/r h ). 

b Ahemaiively. the square root of the expression listed 




TABLE 2-1. -LIST OF EQUAL AMOUNTS 


Notes: 

1 . The symbol in parentheses following the name of the unit is the accepted 
symbol for the unit, if such symbol exists. 

2. An equality sign represents an exact relation established by some 
international standards organization; an approximation sign ( ~ ) indicates that 
the equality is correct only to the number of significant figures given. 

3. The word “mile” alone denotes a U.S. statute mile. The word “second” 
alone denotes an International Atomic Time (TAI) second. 

4. Numerics: 

c a = 0.980 665 
c m = 0.453 592 37 
c 0 = 2.997 924 58 x 10 10 
c P = 1.01325 

5. Abbreviations: 

esu electrostatic system unit 

emu electromagnetic system unit 

Gu Gaussian system unit 

HLu Heaviside- Lorentz system unit 

mksu unrationalized meter-kilogram-second unit 

SIu Systeme International unit 

6. Abbreviations for units should be written without a period and in the 
singular. 

7. The American National Standards Institute (ANSI) recommends that a 
central dot be used to separate the symbols for two units appearing in a product. 

Geometric Quantities 

Length: 

The meter is the distance travelled by an electromagnetic wave in a vacuum 
in (100 /cq) second. 

1 meter (m) = 1 x 10 10 Angstrom (A) = 10 6 micron (n) 

= 10 15 fermi = (1/0.0254) inch (in) = (1/0.3048) foot (ft) 

= (1/1852) U.S. nautical mile. 

[Superseded names: micron - 1 jim; fermi = 1 fm] 

1 X-unit * 1.002 x 10" 13 m 

1 foot (ft) = 12 inch (in) = (1/3) yard (yd) 

= (1/5280) mile (mi) 

1 mile = (463/402.336) U.S. nautical mile 
- (33/38) British nautical mile 

1 astronomical unit (AU) (mean distance between Sun and Earth) 

* 149.6 x 10 9 m 

1 light year * 9.4605 x 10 9 m 

1 parsec * 308.6 x 10 10 m 

* 3.26 light year 

* 2.06 x 10 5 AU 

1 kayser = 1 cm* 1 * a unit of wave number - (wave length) -1 


Area: 

1 square meter (m 2 ) = 10 -4 hectare (ha) = 10 28 bam 
] acre = 4840 square yard 
1 circular mil = (tt/4) x 10 -6 in 2 

Volume: 

1 cubic meter (m 3 ) — 1000 liter = 10 9 lambda (X) 

= 1 stere (st) « 264.172 gallon 

1 gallon = 231 in 3 

1 barrel of oil (bbl) = 42 gallon 

Plane angle : 

1 radian = (180/ir) degree = (200/x) grade (grad) 

Solid angle: 

1 steradian (sr) = (2/ir) spherical right angle 
= (180/x) square degree 

1 sphere = 4x steradian - 8 spherical right angles 
Geometric Quantities and Time 

Time and Frequency: 

1 second (s) = 1 ephemeris second = (1/86400) mean solar day 

* 1.002 737 91 sidereal second 

1 hertz (Hz) = 1 cycle per second (cps), in reference only to a continuous, 
periodic phenomenon 

1 baud = 1 pulse per second 

1 becquerel (Bq) = 1 SIu of activity = 1 /second 
= 1 SIu of radioactive source strength = (10 -t0 /3.7) curie (Ci) 

Speed: 

1 meter per second = (1/0.3048) foot per second (fps) 

= (1/0.44704) mile per hour (mph) 

1 knot (kn) = 1 U.S. nautical mile per hour 

* 1.150 779 mile per hour (mph) 

1 cubic meter per second = 1000 liter per second 
= 2118.88 cubic foot per minute (cfm) 

= 127132.8 cubic foot per hour (cfh) 

1 U.S. gallon per minute (gpm) = (231/1728) cfm 

* 0.003 785 41 cubic meter per minute 

speed of light in vacuo = (cq/ 100) m/s 
Acceleration : 

1 m/s 2 * 100 Gal = 10 11 Edtvds = l/(10c a ) “g” * 3.280 840 ft/s 2 

standard acceleration of gravity (g„) = 1 “g” = I “g unit” 

= 10 c a m/s 2 = (10 c a /0.3048) ft/s 2 
- 32.174 049 ft/s 2 



Mass , Force, and Related Quantities 

Mass: 

1 kilogram mass (kgm or kg) = 1 Slu of mass 
= 0.001 tonne = 0.001 metric ton - 5000 carat 
= 10 9 gamma (7) = l/c m pound mass (lbm or lb) 

= [0.03048/(c a c m )J slug = (7000/c m ) grain 

* 6.0222 x 10 26 atomic mass unit (amu) 

1 slug = 1 geepound 

* 32.174 049 pound mass (lbm or lb) 

1 pound mass = 7000 grain 
Force: 

1 newton (N) = 1 Slu of force = 10 5 dyne 
= 0.001 sthene (sn) = (100/c a ) gram force (gf) 

= [1/(0.3048 c m )] poundal (pdl) 

= [0.1/(c a c m )] pound force (lbf) * 0.224 809 ibf 

1 pound force (Ibf) = 1000 c m gram force (gf) 

* 32.174 049 pdl 

Weight: 

This term has been used to mean either 
“(mass) x (standard acceleration of gravity),” or 
“(mass) x (local acceleration of gravity).” 

The units of weight are the units of force. 

Moment or torque: 

The units of moment or torque are the units of energy. 
Pressure: 

1 pascal (Pa) * 1 Slu of pressure * 1 N«m~ 2 

* 10 -5 bar = (10 _5 /c P ) atmosphere (atm) 

= (0.1/c,) kgf*m -2 = 10 dyne*cm~ 2 

* (0.0076/c p ) torr * (1/133.322) torr 

* (1/3376.85) inch of Hg at 60 *F and g n 

* (1/248.84) inch of H 2 0 at 60 *F and g n 

* 0.000 145 038 lbf/in 2 (psi) 

1 torr = 1mm of Hg at 0 *C and g n 
- (Cp/0.76) millibar (mbar) * 1.33322 mbar 

1 atmosphere = 1 standard atmosphere — 760 mm of Hg 
at 0 *C and g n * 760 torr = 10 5 c P Pa = c P bar 

* 14.695 949 psi 

* 30.0058 in of Hg at 60 *F and g n 

* 33.9324 ft of H 2 0 at 60 *F and g n 

1 in of Hg at 60 *F * 13.5702 in of H z O at 60 # F 

Viscosity 

Viscosity ( dynamic viscosity), 17: 

1 poise (P) = 1 g^cm-^s' 1 = 0.1 Slu of i) 

= 0.1 kg^m'^s” 1 

Kinematic viscosity, v: 

1 stoke (St) = 1 cm 2 *s -1 = 10~ 4 Slu of v ~ 10 -4 m 2 *s -1 


Temperature 

1 Celsius (centigrade) degree (*C) = 1.8 Fahrenheit degree (*F) 

1 degree Rankine (*R) = 1.8 kelvin (K) 

(Note: no degree symbol is used with the symbol for kelvin.) 

(Temperature in °C) = (Temperature in °F - 32)/ 1 .8 

Triple point of water = 273.16 K - 491.688 *R 

Energy and Power 

Energy: 

1 joule (J) = 1 Slu of energy - 1 watt-second (W*s) 

- 1 N*m = 1 Pa*m 3 = 10 7 erg 

= (1/4.184) gram calorie (gcal or cal) 

= (1/4.1868) IT calorie or International Table calorie (cal 1T ) 

= [(0.3048) 2 c m ]-‘ ft*pdl 

* (1/0.042 140 110) fl*pdl 

* (1/1055.06) British thermal unit (Btu) 

- (1/1.355 817 9) ft # lbf 

* 4.587 338 x 10 17 Rydberg 

* (10 l9 /l. 60219) electron volt (eV) 

1 Btu * 778.172 ft»lbf 
* 25037 ft*pdl 
- 4414.37 cal 

Energy of one quantum at wavelength X cm 

* (1.986 484 x 10-23/X) J 

Power: 

1 watt (W) = 1 Slu of power = 1 J/s — 1 N*m/s 
= (1/746) electric horsepower (ehp) 

1 mechanical horsepower (mhp or hp) = 550 ft*lbf/s 

* 745.700 W 
» 0.99960 ehp 

1 ton of refrigeration (U.S.) * 3517 W 

* 200 Btu/minute 
- 840 cal/s 

* 4.7143 ehp 
» 4 frigorie 

Luminous power: [Also see units of Illumination.] 

1 lumen = (1/683) watt of radiation at 540 x 10 12 Hz 
(about 555 nm) 

Quantity 

1 mole (mol) = 0.001 kilomole (kmol) = amount that contains as many entities 
(molecules, atoms, ions, electrons, or other particles or groups of particles) 
as there are atoms in 12 grams of carbon 12. 

When the entity is the molecule, unit molecular weight (traditional name) or 
relative molar mass (more precise name) = 1 gram per mol (g/mol) - 1 
kilogram per kilomole (kg/kmol). 

The number of molecules in 1 kilomole is *6.022 169 x 10 26 (Avogadro’s 
number, N A ). 
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Radioactivity 


Resistance , R: 


2k i 


Activity (source strength): 

1 becquerel (Bq) = 1 nuclear disintegration or transformation 
per second = 1 s' 1 
= 1 SIu of activity = 10 ~ 6 rutherford 

1 curie (Ci) = 3.7 x 10 10 Bq 
Neutron fluence: 

1 nvt = 1 neutron/cm 2 (unless another unit of area is specified) 

Neutron flux (neutron flow rate): 

1 nv - 1 neutron/(cm 2# s) (unless other units of area or time are specified) 
Radiation exposure: 

1 coulomb per kilogram (C/kg) = 1 SIu of exposure 

For X- or 7-radiation, 1 roentgen (R) produces 2.58 x 10 -4 C/kg. 

In air at STP (0 *C, 1 atm), 1 roentgen produces 
1.610 x 10 15 ion pairs per kg, 0.00838 J/kg, 

6.76 x I0 10 MeV/m 3 , 1 esu/cm 3 . 

Absorbed dose of radiation ( kerma , specific energy): 

1 gray (Gy) = I J/kg = 1 SIu of absorbed dose = 100 rad or rd 

Radiation dose equivalent ( equivalent dose of radiation): 

1 sievert (Sv) = 1 J/kg = 1 SIu of dose equivalent = 100 rem (roentgen 
equivalent mammal) 

1 rep (roentgen equivalent physical) = dosage that produces 0.00838 J/kg 
in mammalian tissue 

Electricity and Magnetism 

Current, I: 

1 ampere (A) = 1 SIu of / = (Cq/10) statampere = 0.1 abampere 
1 statampere = 1 Gu of / = V(4 t) HLu of 7 

Charge, Q: 

1 coulomb (C) = I SIu of Q - 1 mksu of Q 
= (Co/10) statcoulomb = 0.1 abcoutomb * (1/96487) faraday 

1 electron charge (e) = 1.602 1917 x 10 " ,9 C 
Capacitance, C: 

1 farad (F) = I SIu of C = 1 mksu of C 
= Cq x 10 -9 statfarad = 10 -9 abfarad 

1 statfarad = 1 Gu of C = 4x HLu of C 

Permittivity, e = e,* 0 .* [t r = relative permittivity (dimensionless)] 

Permittivity of a vacuum, t 0 
= (10 n /(4xcj*)] farad per meter (F/m), in SI 
= (10 ll /cJ) F/m, in mks system = 1 stat farad/cm 
= 1 Gu of e = 1 HLu of t = (1/cJ) abfarad/cm 


1 ohm (Q) = 1 SIu of R = 1 mksu of R 
= (10 9 /Cq) statohm = 10 9 abohm 

1 statohm = 1 Gu of R = [l/(4x)] HLu of R 
Conductance, G: 

1 siemens (S) = I SIu of G - 1 mksu of G — (ohm) -1 
= Cq x 10 -9 esu of G = 10 -9 emu of G 

- 1 mho (superseded name) 

1 esu of G = 1 Gu of G = 47r HLu of G 
Resistivity , p q : 

1 ohm-meter (fl«m) = 1 SIu of p q - 10 R microhm cm 
= [48 x 10 8 /(2.54ir)] ohm per circular mil foot 
= (12 x 10 8 /2.54) ohm per square mil foot 

Inductance, L: 

1 henry (H) = 1 SIu of L - 1 mksu of L = 10 9 abhenry 
= (I0 9 /Cq) stathenry 

1 stathenry = 1 Gu of L - [1/(4t)J HLu of L 
Electric flux, V : 

I coulomb (C) = l SIu of 'k = 4x mksu of 'k 
= 0.4 t emu of 'k = 0.4r Cq esu of ^ 

1 esu of * = 1 Gu of * = [W(4i)] HLu of * 

Electric displacement , D: 

1 coulomb per square meter (C/m 2 ) = 1 SIu of D 

— 4ir mksu of D = 4xc 0 x 10 ~ 5 esu of D 

— 4t x 10 -5 emu of D 

1 esu of D = 1 Gu o f D = [1/V(4x)] HLu of D 
Electric polarization, P: 

1 coulomb per square meter (C/m 2 ) = 1 SIu of P 
= 1 mksu of/ > = c 0 xlO -5 statcoulomb/cm 2 
= 10 -5 abcoulomb/cm 2 

1 statcoulomb/cm 2 = 1 Gu of P = V(4t) HLu of P 

Electric susceptibility per unit volume, \ t v : 

1 SIu of x, v = [I/(4x>] esu of x, (V = 1 HLu of x rv 

1 esu of x*, v — 1 mksu of x«, v = 1 emu of x^.v 

- 1 Gu of Xt.v 

Electric moment, M q : 

1 coulomb-meter (C*m) = 1 SIu of M q = 1 mksu of M q 
= 10 c 0 esu of M q - 10 emu of M q 

1 esu of M q = 1 Gu of M q = \/(4x) HLu of M q 

Electromotive force ( electric potential difference), E: 

1 volt (V) = 1 SIu of E = 1 mksu of E 
= (IO^/Cq) statvolt = 10 8 abvolt 

1 statvolt = 1 Gu of £ = [1/V(4x)l HLu of E 


217 



Electric field strength, 6; 

1 volt per meter (V/m) = 1 SIu of 8 = 1 mksu of 8 

- (lO 6 /^) statvolt/cm = 10 6 abvolt/cm 

1 statvolt/cm = 1 Gu of 8 = [l/V(4x)J HLu of 8 

Permeability , p = [ p r = relative permeability (dimensionless)] 

Permeability of a vacuum, p$ 

= 4x x 10" 7 henry per meter (H/m) in SI 
= 10~ 7 H/m in mks system = I abhenry/cm 
= (1 /Cq) stathenry/cm = 1 Gu of p — 1 HLu of p 

Magnetic pole strength, m *: 

1 weber (Wb) = 1 Slu of m* = 1 volt-second 
= 1 ampere-henry = (4x)“ 1 mksu of m* 

= [ 10 8 /(4irc 0 )] statpole = [10*/(4x)] abpole 

1 abpole = 1 Gu of m* = V(4x) HLu of m* = (l/c^) statpole 
Magnetic flux, 4>: 

1 weber (Wb) = 1 SIu of * = 1 volt-second 
= 1 ampere-henry = 10* maxwell (Mx) = 1 mksu of * 

= (lO 8 /^ esu of * = 10* emu of * 

1 emu of ♦ * 1 maxwell (Mx) = 1 line - 1 Gu of 
= Il/V(4x)] HLu of* = (1 /Cq) esu of * 

Magnetic induction, B: 

1 tesla (T) = 1 Wb/m 2 * 1 SIu of B - 1 mksu of B 

- I0 4 gauss = 10 4 Mx/cm 2 

1 gauss = 1 emu of B = ( 1/Cq) esu of B * 1 Gu of B 
= [1/V(4x)] HLu of B 

Magnetic polarization, J: 

1 tesla (T) = 1 Wb/m 2 = 1 SIu of J 

= [1/(4t)] mksu of J = [10 4 /(4t)] emu of J = 10 9 gamma (y) 

1 emu of J = (1/Cq) esu of J = 1 Gu of J * V(4x) HLu of J 

Magnetic susceptibility per unit volume, x m , v ; 

1 Slu of x m ,v = * HLu of x m , v = [l/(4x)] emu of x m>v 

1 emu of x ffl ,v * 1 esu of x,„.v * 1 mksu of x m v 
= 1 GU Of Xm,v 

Magnetic moment, M m : 

1 weber-meter (Wb*m) = 1 SIu of M m - (l/(4x)] mksu of M m 
= [10 ,0 /(4x)3 emu of M m 

1 emu of M m = 1 Gu of M m = V(4x) HLu of M m 
= (1/cq) esu of M m 

Magnetomotive force, 

1 ampere-turn = 1A = 1 SIu of T = 4 t mksu of J * 0.4x emu of J 

1 gilbert = 1 emu of $ * Cq esu of $ * 1 Gu of $ 

= [1/ (4x)] HLu of 5 

Magnetic field strength, H: 


1 oersted (1 gauss before 1930) = 1 emu of H 
= 1 Gu of H = c 0 esu of H = [ I /%/ (4x)] HLu of H 

Reluctance, <R: 

1 ampere-turn per weber = 1 SIu of (R = 4x mksu of (R 
= 4x x 10” 9 emu of (R = I0 9 /(4x) gilbert per maxwell 

1 gilbert per maxwell = 1 emu of <R = c^ esu of <R 
= 1 Gu of <R = 1 HLu of (R 

Radiation 

Physical quantities in this field are measured in units of the dimensions 
indicated in Table 1-5; namely, units of energy, power, area, volume, time, 
and solid angle. 

Illumination 

Luminous intensity, / v : 

1 candela (cd) = 1 SIu of 7 V = luminous intensity of a source that emits 
monochromatic energy of frequency 540 x 10 12 Hz (about 555 nm 
wavelength) and that has a radiant intensity of (1/683) wan per steradian * 1 
candle 

Luminous flux ( luminous power), * v ; 

1 lumen (lm) = 1 SIu of * v — luminous flux emined per steradian by a 
1 -candela source = 1 cd*sr 

Luminance (of a source), L^\ 

1 cd/m 2 * 1 SIu of L, * 1 nit (nt) = 10 -4 stilb (sb) 

= x apostilb (asb) * 10 _4 x lambert (L) 

1 apostilb (asb) = 10 " 4 lambert (L) = (0.3048) 2 footlambert 

Luminous flux density (of, through, or on a surface): 

1 lumen per square meter (Im/m 2 ) 

= 1 SIu of luminous flux density 

Luminous flux density, M v , of a surface (luminous exitance, 
luminous emittance): 

I lm/m 2 = 1 SIu of M y « (0.3048) 2 lm/ft 2 

Luminous flux density, E v , on a surface (illuminance, illumination): 

1 lm/m 2 * 1 lux * 1 SIu of E v = 10 ~ 4 phot (ph) 

■ (0.3048) 2 footcandle 

1 footcandle = 1 lm/ft 2 
Luminous energy, W v : 

1 lumen-second (lm*s) = 1 talbot = 1 SIu of W v 

Luminous efficacy, K v : 

1 lumen per watt (Im/W) « 1 SIu of K w 

Maximum luminous efficacy, R v mMX * (1/683) lm/W 

Sound 

For psychoacoustical units, see Example 2.8h. 


1 ampere per meter (A/m) = I SIu of H = 4x mksu of H 

= 0.004x oersted (Oe) 



TABLE 2-2. -SOME RELATIONSHIPS AMONG COHERENT UNITS 


IN = 1 kg»m/s 2 = 1 Wb»A/m = 1 Wb 2 /(H»m) = 1 C»V/m = 1 C 2 /(F»m) 

1 dyne = 1 g*m/s 2 = 1 (statcoulomb) 2 /(cm»stathenry) = 1 (abcoulomb) 2 /(cm»abhenry) 

I pdl = 1 lb»ft/s 2 
1 Ibf = 1 slug«ft/s 2 = g n * 1 Ibm 
1 gf = «„* 1 g 

1J = 1 W*s = 1 N*m = I V*C = I A»Wb = 1 Pa^m 3 

1 erg = I dyne*cm = 1 statvolt*statcoulomb = 1 abfarad*abcoulomb 

1 second = 1 F*ft = 1 statfarad*statohm — 1 abfarad # abohm 
= t (HLu of C)*(HLu of R) 

= 1 H/G = 1 stathenry/statohm = 1 abhenry/abohm = I (HLu of L)/(HLu of R) 

= 1 Wb/V = 1 C/A 

1 F *= 1 C/V; 1 statfarad = 1 statcoulomb/statvolt; 

1 abfarad = 1 abcoulomb/abvolt; 1 (HLu of C) = 1 (HLu of Q)/(HLu of E) 

1 H = 1 Wb/A; 1 stathenry = 1 (esu of 4>)/statampere; 1 abhenry = I (emu of 4>)/abampere 
1 (HLu of L) = c 0 (HLu of *)/(HLu of I) 


Symbols for units used in this table: 


A 

ampere 

g 

gram mass 

lb 

pound mass 

pdl 

poundal 

C 

coulomb 

gf 

gram force 

lbf 

pound force 

s 

second 

cm 

centimeter 

H 

henry 

m 

meter 

V 

volt 

F 

farad 

J 

joule 

N 

newton 

w 

watt 

ft 

Q 

foot 

ohm 

kg 

kilogram mass 

Pa 

pascal 

Wb 

weber 


Constants used in this table: 

Cq = 2.99792458 x 10 10 

g n = (10 c a )m/s 2 = (c,/0. 03048) ft/s 2 * 32.1740 ft/s 2 
c, = 0.980665 


TABLE 2-3 -PREFIXES OF THE 
SYSTEME INTERNATIONAL 


Power 
of 10 

Prefix 

Symbol 

Power 
of 10 

Prefix 

Symbol 

(1 

deka 

da) 

("I 

deci 

d) 

(2 

hecto 

h) 

(-2 

centi 

c) 

3 

kilo 

k 

-3 

milli 

m 

6 

mega 

M 

-6 

micro 


9 

giga 

G 

-9 

nano 

n 

12 

tera 

T 

-12 

pico 

P 

15 

peta 

P 

-15 

femto 

f 

18 

exa 

E 

-18 

atto 

a 


Note: The prefixes enclosed in parentheses are less 
desirable. Powers that are multiples of 3 are preferred. 





TABLE 5-1. -STATION LOCATIONS AND WEIGHTS FOR AVERAGING 


Averaging for linear interval 0 < x < 1 

Averaging in a circular duct, in interval 0 < r < 1 


Number 

of 

Method 













stations 

Centroid of equal areas 

Newton-Cotes 


Chebyshev 


Gauss 


n 

X 

r 

H' 

X 

r 

w 

X 

r 

w 

X 

r 

Vi’ 

2 




(b) 

(b) 

(b) 

(e) 

(e) 

(e) 

<e) 

(e) 

(e) 


0.2500 

.7500 

0.5000 

.8660 

1/2 

0 

1 

0 

1 

1/2 

0.2113 

.7887 

0.4597 

.8881 

1/2 

0.2113 

.7887 

0.4597 

.8881 

1/2 

3 




<C) 

(c) 

(c) 








0.1667 

0.4082 


0 

0 

0.1667 

0.1464 

0.3827 


0.1127 

0.3357 

0.2778 


.5000 

.7071 

1/3 

0.5 

0.7071 

.6667 

.5000 

.7071 

1/3 

.5000 

.7071 

.4444 


.8333 

.9129 


1 

1 

.1667 

.8536 

.9239 


.8873 

.9420 

.2778 

4 




(d) 

(d) 

(d) 








0.1250 

0.3536 


0 

0 

0.1250 

0.1027 

0.3203 


0.0694 

0.2635 

0.1739 


.3750 

.6124 

1/4 

0.3333 

0.5774 

.3750 

.4062 

.6373 

1/4 

.3300 

.5745 

.3261 


.6250 

.7906 

.6667 

.8165 

.3750 

.5938 

.7706 

.6700 

.8185 

.3261 


.8750 

.9354 


1 

1 

.1250 

.8973 

.9473 


.9306 

.9647 

.1739 

5 

0.1000 

0.3162 


0 

0 

0.0778 

0.0838 

0.2894 


0.0469 

0.2166 

0.1185 


.3000 

.5477 


0.25 

0.5000 

.3556 

.3127 

.5592 


.2308 

.4804 

.2393 


.5000 

.7071 

1/5 

.50 

.7071 

.1333 

.5000 

.7071 

1/5 

.5000 

.7071 

.2844 


.7000 

.8367 


.75 

.8660 

.3556 

.6873 

.8290 


.7692 

.8771 

.2393 


.9000 

.9487 


1 

1 

.0778 

.9162 

.9572 


.9531 

.9763 

.1185 

6 

0.0833 

0.2887 


0 

0 

0.0660 

0.0669 

0.2586 


0.0338 

0.1838 

0.0857 


.2500 

.5000 


0.2 

0.4472 

.2604 

.2887 

.5373 


.1694 

.4116 

.1804 


.4167 

.6455 

1/6 

.4 

.6325 

.1736 

.3667 

.6055 

1/6 

.3807 

.6170 

.2340 


.5833 

.7638 

.6 

.7746 

.1736 

.6333 

.7958 

.6193 

.7870 

.2340 


.7500 

.8660 


.8 

.8944 

.2604 

.7113 

.8434 


.8306 

.9114 

.1804 


.9167 

.9574 


1 

1 

.0660 

.9331 

.9660 


.9662 

.9830 

.0857 

7 

0.0714 

0.2673 


0 

0 

0.0488 

0.0581 

0.2410 


0.0254 

0.1595 

0.0647 


.2143 

.4629 


0.1667 

0.4082 

.2571 

.2352 

.4849 


.1292 

.3595 

.1399 


.3571 

.5976 


.3333 

.5774 

.0321 

.3381 

.5814 


.2971 

.5450 

.1909 


.5000 

.7071 

1/7 

.5000 

.7071 

.3238 

.5000 

.7071 

1/7 

.5000 

.7071 

.2090 


.6429 

.8018 


.6667 

.8165 

.0321 

.6619 

.8136 


.7029 

.8384 

.1909 


.7857 

.8864 


.8333 

.9129 

.2571 

.7648 

.8745 


.8708 

.9331 

.1399 


.9286 

.9636 

i 


1 

1 

.0488 

.9419 

.9705 


.9746 

.9872 

.0647 

n 

X 

y v'T* 

i r 

W 

X 

r 

w 

X 

r 

w 

jr 

r 

i 

w 


Centroid of equal areas 

Newton-Cotes 

Chebyshev 

Gauss 




TABLE 5-1. -Concluded. 


Number 

Method 

or 













stations 

Centroid of equal areas 

| Newton-Cotes 


Chebyshe 

(al 

V 


Gauss 


n 

X 

r 

w 

X 

r 

w 

X 

r 

w 

X 

r 

w 

8 







(f) 

<f) 

(0 





0.0625 

0.2500 


0 

0 

0.0435 

0.0513 

0.2266 


0.0199 

0.1409 

0.0506 


.1875 

.4330 


0.1429 

0.3780 

.2070 

.2031 

.4507 


.1017 

.3189 

.1112 


.3125 

.5590 


.2857 

.5345 

.0766 

.2969 

.5449 


.2372 

.4871 

.1569 


.4375 

.6614 


.4286 

.6547 

.1730 

.4487 

.6698 


.4083 

.6390 

.1813 


.5625 

.7500 

I/O 

.5714 

.7559 

.1730 

.5513 

.7425 

1 to 

.5917 

.7692 

.1813 


.6875 

.8292 


.7143 

.8452 

.0766 

.7031 

.8385 


.7628 

.8734 

.1569 


.8125 

.9014 


.8571 

.9258 

.2070 

.7969 

.8927 


.8983 

.9478 

.1112 


.9375 

.9682 


1 

1 

.0435 

.9487 

.9740 


.9801 

.9900 

.0506 

9 

0.0556 

0.2357 


0 

0 

0.0349 

0.0442 

0.2103 


0.0159 

0.1262 

0.0406 


.1667 

.4082 


0.1250 

0.3536 

.2077 

.1995 

.4466 


.0820 

.2863 

.0903 


.2778 

.5270 


.2500 

.5000 

-.0327 

.2356 

.4854 


.1933 

.4397 

.1303 


.3889 

.6236 


.3750 

.6124 

.3702 

.4160 

.6450 


.3379 

.5813 

.1562 


.5000 

.7071 

1/9 

.5000 

.7071 

-.1601 

.5000 

.7071 

1/9 

.5000 

.7071 

.1651 


.6111 

.7817 


.6250 

.7906 

.3702 

.5840 

.7642 


.6621 

.8137 

.1562 


.7222 

.8498 


.7500 

.8660 

-.0327 

.7644 

.8743 


.8067 

.8982 

.1303 


.8333 

.9129 


.8750 

.9354 

.2077 

.8005 

.8947 


.9180 

.9581 

.0903 



.9718 


1 

1 

.0349 

.9558 

.9776 


.9841 

.9920 

.0406 

10 







(g) 

(g) 

(g) 





0.05 

0.2236 


0 

1 0 

0.0319 

0.0419 

0.2046 


0.0130 

0.1142 

0.0333 


.15 

.3873 


! o.mi 

! 0.3333 

.1757 

.1564 

.3954 


.0674 

.2597 

.0747 


.25 

.5000 


.2222 

.4714 

.0121 

.2500 

.5000 


.1603 

.4004 

.1095 


.35 

.5916 


.3333 

.5774 

.2159 

.3436 

.5862 


.2833 

.5323 

.1346 


.45 

.6708 

1/10 

4444 

.6667 

.0645 

.4581 

.6768 

i /in 

.4256 

.6524 

.1478 


.55 

.7416 


.5556 

.7454 

.0645 

.5419 



.5744 

.7579 

.1478 


.65 

.8062 


.6667 

.8165 

.2159 

.6564 


■ 

.7167 

.8466 

.1346 


.75 

.8660 


.7778 

.8819 

.0121 



1 

.8397 

.9164 

.1095 


.85 

.9220 


.8889 

.9428 

.1757 

.8436 


■ 

.9325 

.9657 

.0747 


.95 

.9747 


1 

1 

.0319 

.9581 


■ 

.9870 

.9935 

.0333 

n 

X 

r 

w 

X 

r 

H’ 

X 



X 

r 

w 


Centroid of equal 

areas 

Newton-Cotes 

Chebyshev 

Gauss 


NOTES 


(a) All measurements of equal weight (e) 0.2-0. 8 rule 

(b) Trapezoidal rule (f) Two 4-station intervals 

(c) Parabolic rule (Simpson’s rule) (g) Two 5-station intervals 

(d) Three-eighths rule 








Subject Index 

References are to the section number except where the prefix E indicates example number. 1 


absolute magnitude, E2.8/, E2.11a 
absolute system of units, 2.6 
acceleration due to gravity, 2.1, 5.4 
accelerometer, 5.1, 6.1, 6.3 

airplane, E7.23a 
accuracy, specification of, 3.34 
acoustic capacitance, 10.1 
inertance, 10.1 
resistance, 10.1 
units, E2.8e,g,h,j 
acuity, vernier, 5.19 
visual, 5.19 

airplane acceleration measurement, 5.1, E7.23a 

albedo measurement, 3.2 

Alnico, 5.5 

ammeter, E5.17a 

amounts, list of equal, 2.1 

Ampere’s law, 1.1 

amplitude ratio, 7.12, 7.20 

analytic function, definition of, 3.19 

anemometer, hot-wire, E1.5d, E5.16b 

ANSI, 2.1 

approximation of intricate functions, 4.22 

arithmetic mean, 3.13 

atomic mass, 1.1 

audiometry, errors in, 3.2 

audio units, E2.8J 

auditor, standard, E2.8h 

average deviation, ideal, 3.17 

practical, 3.19 
See also error. 

average deviation of the error, 3.22 
average error, 4.5 
averaging, nonlinear, 5.16 

spatial, 5.10, 5.13 

background error, 3.26 
backlash, 5.6 
balance, weighing, 5.4 
balance weights, E3.31 
ballistic galvanometer, 7.19 
method, 8.1 
pendulum, 7.19 
band, working, 5.1 
bandwidth, definition of, 7.20 
bandwidth at half maximum, 5.19, 7.21 
bars, error, 4.7 
bearing friction, 5.6 

*This index does not include physical quantities exclusively listed in 
Tables 1-1 to 1-5, and Table 2-1. 


bel, E2.8e 
bellows, 6.2, 8.4 
Bernoulli’s principle, 3.8 
blackbody, E5.11b 
bolometer, 5.18 

bolometric magnitude, E2.8/, E2.11a 
bottle resonator, 6.7 
boundary-layer-type flowmeter, E5.8e 
bridge, Wheatstone, El. 4c 
Brownian motion, 5.19 
Buckingham, 1.3 

bulk velocity measurement, E5.10a, E5 . 1 Id-f, E5.15a,b 
burst-diaphragm method, 8.1 

calibration correction curves, 3.33 
factor, 1.6 

calorimetric bomb, E5. 1 la 
calorimetric flowmeter, E5.8e, E5.11e 
candela, 2.7 
cantilever, 6.2, 8.4 
capacitance, acoustic, 10.1 

thermal, 10.1, 10.5 
capacitors, 7.12 
central limit theorem, 3.29 
CGPM, 1.3 
cgs system, 2.2 

charge measurement, error in, 3.2 
Chebyshev criterion for best fit, 4.3 

criterion for linearization, 4.9 
integration, 5.13 
polynomials, E4.16a, 4.21 
chemical balance, E3,llc 
chopper, light, E3.11f 
choppers, 5.21 
chord-based straight line, 4.9 
CIPM, 2.1 

coefficient, correlation, 4.12 
probability, 3.16 
coherent equations, 2.10 
coherent system of units, 2.1 
combination of errors, 3.28 

of steady-state dynamic responses, 7.5 
compressibility factor of H 2 , E4.22a 
of N 2 , E4.22b 

conductivity of electrolyte, measurement of, 3.2 
confidence limit, 3.18 

consistency, internal and external, 3.30, 3.32 
convective heat transfer, E1.5d, E3.28h, E5.11c 
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dynamic vibration absorber, E9.3c, 10.2 
dynamometer, 5. 3-5. 6, 6.1 


comer frequency, 7.12, 8.3 
correction, definition of, 3.3 
corrections, summation of, 3.9, 3.28 
correlation coefficient, 4.12 
index, 4.12 

cosmic radiation, 3.2, 3.26 
Coulomb’s law, 1.1 
counting rate, 3.26 
creep, 5.6 

critical damping, 7.14 

crystal, piezoelectric, El. 6b, E4.1a 

cubic spline, 4.18 

current measurement, null method, 5.18 
noise, 5.19 

curvature, radius of, 7.13, 7.21, 10.3 

cylinder, heat transfer to, E3.28h See also hot wire. 

D (dissipation factor), 7.12 

damping, 7.14 

damping factor, 7.15 

d’Arsonval galvanometer, 6.3, 8.4 

dead-time error, 3.26 

decibel, E2.8e 

decile, 3.20 

decrement, logarithmic, 7.15, 8.1 
delay time, E10.6c 
density of a liquid, E4.16c 
derivative, partial, approximation of, 3.9 
deviation, average, 3.19 
rms, 3.19 
standard, 3.19 
See also error. 

deviation of the deviation, 3.22 
diaphragm, natural frequency of, 8.4 
differential equation, least-squares solution for, 4.17 
diffusion coefficient, 5.19 
digits, significant, E3.22a, 4.20 
dimensional analysis, 1.5 
dimensional constants, 1.1, 2.7 
dimensionless numbers, 1.3, 2.9 
dimensionless units, 2.8 
dimensions of errors, 4.5, 4.6 
discrimination, frequency-response, 7 .23 
dispersion, ideal measures of, 3.17 
index of, 3.18 
practical measures of, 3.19 
uncertainty in, 3.22 

dispersion of errors, 3.15, 3.16, 3.17, 3.19 
dissipation factor, 7.12 

distribution of random errors , 3.16 See also dispersion . 
dither, 5.6 

drag coefficient , E 1 . 6a 
measurement, 3.2 
droplet size measurement, 3.2 
dry friction, 5.6 


eccentricity, mechanical, effect of, E3.11a,b 
Einstein, 5.19 
elasticity, modulus of, 5.6 
elbow, pipe, effect of, 5.13 
electrical units, 2.1, 2.4 
electricity, physical quantities in, 1.1 
electrobalance, 3.11, 5.5, 6.3 
emf measurement, errors in, 3.2 
empirical coefficients, 1.6 
empirical graphs and equations, 4.1 
emu system, 2.4 
energy transfer, 5.17 
equal amounts, list of, 2.1 
equations, 4.3 
error, definition of, 3.3 
fractional, 3.5 
limit of, 3.7 
measures of, ideal, 3.17 
measures of, practical, 3.19 
sources of, 3.2 
error bars, 4.7 

distribution, 3.16 
frequency distribution, 3.16 
of error, 3.22 

of indirectly measured quantities, 3.31 
of parameters, 4.16 
rectangles, 4.7 
errors, combination of, 3.28 
esu system, 2.4 
events, rate of, 3.26, 3.27 
exponential, approximation of, 4.3, 4.17 
exponential forcing function, 7.8, 7.16 
extensometer, E3.11f. See also strain gauge, 
external consistency, 3.30, 3.32 

factor, calibration, 1.6 
Faraday constant, El.l 
Faraday’s law, El.l 
FET (field-effect transistor) noise, 5.19 
figure of merit of galvanometer, 8.4 
first-order differential equation, least-squares 
solution, 4.17 
Fisher, 4.12 
flicker noise, 5.19 

flow, two-phase, as source of error, 3.2 
flow measurement, errors in, 5.8 
of a river, 3.2, 5.13 
flowmeter, boundary-layer-type, E5.8e 
calorimetric, E5.8e, E5.11e 
impact disk, E5.16c 

head-type, E1.4g, E5.8e, E5.16a E5.17d 
head-type, exact formula, 4.22d 
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Thomas-ty pe , E5 . 1 1 e 
turbine-type, E5.17d 
ultrasonic, E5.1 If 
flowmeter nozzle, E5.8e 
fluid density measurement, errors in, 3.2 
fluid velocity measurement, errors in, 3.2 
force measurement, 5. 3-5. 6 
forcing function, definition of, 7.5 
some types of, 7.6 
Fourier's equation, 7.22 
fps and fpsg systems, 2.2 
fractional error, 3.5, 3.19 
free-molecule regime, E3.13d 
French curve, 4.11, 4.18 
frequency, corner, 7.12, 8.3 

natural, 6.13, 8.4, 9.2 
resonant, 7.20 

frequency discrimination, 7.23 

distribution of errors, 3.16 
distributions, some types of, 3.18 
meter, 3.27 
ratio, 7.20 
friction, dry, 5.6 

galvanometer, 6.3 
gas analyzer, E1.4f 
gas constant, El.l 

gas flow rate, E2.11d, E3.13a,b, E4.22d 

gas law, El.l 

Gaussian distribution, 3.18 

system of units, 2.4 

Gauss's criterion for best fit, 4.3, 4.5, 4.10 
criterion for consistency, 3.30 
law, El.l 

method of integration, 5.13 
Geiger-Mueller tube, 3.2, 3.27 
generalized coordinates and forces, 9.3 
generation-recombination noise, 5.19 
generator, noise, 5.19 
geometric mean , 3.13 
Giorgi, 2.6, 2.7 
gradient, pressure, 7.17 

velocity, E5.13d, 7.18 
Grand Mean, 3.30, 3.32 
graphs, 4.1, 4.2, 4.7 
gravity, standard, 3.1, 5.4 

half-life, 7.7 
harmonic mean, 3.13 
heat, physical quantities in, 1.1 
units of, 2.1, 2.3 
heat flow measurement, 3.2 
transfer, E3.28h, 6.12 
See also hot wire, calorimetric flowmeter. 
Heaviside-Lorentz system, 2.5 


Heaviside operator, 7.4 
helical spring, 8.4 
Helmholtz resonator, 6.7 
high-vacuum conditions, E3.13d 
hotwire, E1.4f, E1.5d, El. 6c 
hot-wire anemometer, E1.5d, E5.16b 
hum, 5.7 
hysteresis, 5.6 

IGFET (insulated-gate field-effect transistor) noise, 5. 
illumination, 1.1, 1.2 
impact disk, force on, El. 5a 
impact disk flowmeter, E5.16c, 7.18 
impact force, El. 5a 
impedance, flow-, calibration of, 6.11 
impulse, 7.11, 7.19 
inaccuracy, descriptions of, 3.6 
independent variables, definition of, 3.28 
index of correlation, 4.12 
of dispersion, 3.18 
inductors, 7.12 

industrial process instruments, 5.0 

infinite line, 10.6 

input noise, equivalent, 5.19 

installation requirements, consideration of, 5.23 

instantaneity, definition of, 6.0 

instrument selection, factors in, 3.35 

integration of events, 3.27 

integration, spatial, 5.13 

interference, radiofrequency, 5.7 

internal consistency, 3.30, 3.32 

International set of units, 2.6 

interpolation of data, 4.19 

ion gauge, 3.2, E5.17f 

ionosphere, 3.2 

JFET (junction field-effect transistor) noise, 5.19 
Johnson noise, 5.19 
journal bearings, 5.6 

kelvin, 2.3 
keying, 8.1 

kinematic viscosity of oils, 4.8 

Lagrange's equation, 9.3 

interpolation formula, 4.19 
laminar profile, 5.10 
Laplace transform, 7.4 
laser, in measurement, 3.2 
laser noise, 5.19 

leak, pneumatic, calibration of, 6.11 
level, definition of, E2.8g 
loudness, E2.8h 
limit of error, 3.6, 3.7 
line, long, 10.6 
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linear approximations, 4.8 
linearity, 4.9 
linearization, 4.8 
linearized constructions, 4.8 
logarithmic decrement, 7.15, 8.1 
mean, 3.13, E3.21b,c 
log log graph paper, 4.8 
Lorentz system of units, 2.5 
loudness, 1.2, E2.8h 
loudness level, E2.8h 
luminance discrimination, 5.19 

magnetic properties, errors in measuring, 3.2 
units, 2.1, 2.4 

magnetism, physical quantities in, 1.1 

magnitudes, stellar, E2.8/, E2.11a 

maintenance requirements, consideration of, 5.23 

manganin, resistivity of, E3.28m 

manometer, 6.4, 6.5 

mass correction for springs, 6.2 

mean, geometric (logarithmic), E3.21b,c 

mean of means, 3.30 

means, some types of, 3.13 

measures of dispersion, ideal, 3.17 

practical, 3.19 

mechanics, physical quantities in, 1.1 
median, 3.13 

median error distribution, 3.18 
“method of least squares”, 3.19, 4.10 
microphone, probe, 10.6 
minimum-error-based straight line, 4.9 
mks system, 2.6 
MKS system, 2.6 
modulus of elasticity, 5.6 
molar mass, relative, 1.1 
mole, 2.7 

molecular mass (molecular weight), 1.1, 2.7 

molecular speed, E3.13d 

most probable value, ideal, 3.18 

practical, 3.19 

most probable value of a function, 3.28 

of frequency of events, 3.26, 3.27 

natural frequency, definition of, 6.13 
neper, E2.8f 
nephelometer, E5.8d 
nephelometry, errors in, 3.2 
neutron flux, effect of, 5.8 
Newton-Cotes method, 5.13 
Newton’s law, El.l, 6.1 
NiSpan C, 5.6 
noise, 5.7, 5.19-5.21 
white, 8.3 
noise factor, 5.19 

figure, E2.8i, 5.19 


generator, 5.19 
power, E2.8i, 5.19 
resistance, 5.19 
temperature, 5.19 
thermometer, 5.19 
noisiness, 1.2 
nonlinear averaging , 5.16 
indication, 3.8 
nonlinearity, 4.9, 5.0 
normal equations, 3.31, 4.10, 4.15 
Norton’s theorem, 5.18 
nozzle, sonic, E3.28/ 
nozzle-type flowmeter, E5.8e 
null detector, 1.4 
null measurements, 5.18 
numerical equations, 2.11, 2.12 
Nusselt number, E1.5d 
Nyquist diagram, 7.12, 7.20 
noise, 5.19 

observer, standard, 1.2 
odds of occurrence of errors, 3.18 
“of a single observation”, 3.19 
“of the mean (or median)”, 3.19 
oil, viscosity of, 4.8 
operational calculus , 7.4 
operator, differential, 7.4 
optics, physical quantities in, 1.1 
units of, 2.1 
organ pipe, 6.7 

orifice, flow through, E2.11d, E4.22d 
orifice-type flowmeter, E5 .16a, E5 . 1 7d 
overbar, meaning of, E3.31 

parabola, quadratic, response to, 7.10, 7.15 
parabolic rule, 5.13 
parallax, 5.2 

parallel combination of subsystems, 7.4 

parallel electrical circuit, 6.9 

partial derivative, approximation of, 3.9 

Peclet number E 1 4f , E 1 . 5d 

pendulum, E1.4e, El. 5b, El. 6b, E2.11e, 8.4 

pentode noise, 5.19 

percentage error, 3.5. See also fractional error. 

percentiles, 3.20 

perception, quick, 5.1 

permeability of air, 3.1 

permittivity of air, 3.1 

pH, definition of, E2.8k 

phase angle, 7.12, 7.20 

phon, E2.8h 

photometry, errors in, 3.2 
piecewise curve fit, 4.11,4.18 
piezoelectric crystal, El. 6b, 3.2 
Pi theorem, 1.3 
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pitot-static tube, 3.2, E3.13b 
Pi-zero theorem, 1 .4 
Planck’s constant, El.l 
law, El.l 

pneumatic systems, 6.6, 6.7, 10.1, 10.5, 10.6 
pointer-and-scale instruments, 5.1 
Poiseuille’s law, 6.6, 10.6 
Poiseuiile-type flow, 10.6 
Poisson distribution, 3.26 
population, statistical, 3.14 
potentiometer, El .4a,b 
power, electric, average, E5.15c 
power balance equation, 9.1 
transfer, 5.17, 5.18 
practice, recommended, 3.34 
Prandtl number, E1.5d 
prefixes, SI, 2.7 

pressure gauge, specification of, E5.7b, 6.1 
measurement, null method, 5.18 
probabilities of errors, 3.18 
probability coefficient, 3.16 
graph paper, 4.8 
probable error, ideal, 3.17 

practical, 3,19 
See also error. 

probable error of the error, 3.22 
probe microphone, 10.6 
profile, effect of, 5.9, 5.10 
laminar, 5.10 
spatial, 5.9, 5.12, 5.13 
spatial, development of, 5.10, 5.15 
turbulent, 5.10 
velocity, E5.15a 
profile measurement, 5.12 
proportionalities in physics, 1.1 
psychophysical quantities, 1.2, E2.8h 
pulse, response to, 7.3, 7.11 

See also impulse; step change, 
triangular, 7.11 
pumping, by ion gauge, E5.17f 
pyrometer, radiation, 3.2, E5.16d 
pyrometry, errors in, 5.8 

Q (storage factor), 7.12 
quadratic, approximation of, 4.9 
quadratic forcing function, 7.10, 7.18 

radiation, cosmic, 3.26 

measurement of, 3.2 
physical quantities in, 1.1 
radiation pyrometer, E3.28i, E5.16d 
radioactivity, measurement of, 3.2, 3.26, 7.7 
units of, 2.1 

radiofrequency interference , 5.7 

radius of curvature, minimum, 7.13., 7.21, 10.3 


ramp, as forcing function, 7.9, 7.17 
response to, 7.9, 7.17 
test by, 8.2 

random error distributions, 3.16, 3.18 
of a function, 3.28 
random errors, 3.10, 3.12 
random events, counting of, 3.26 
integration of, 3.27 
range, definition of, 3.4 
ratemeter, 3.27 

rate of events, measurement of, 3.26, 3.27 
rationalized units, 2. 5-2. 7 
Rayleigh’s method, 9.2 
readability, 5.0, 5.2 
recommended practice, 3.34 
rectangles, error, 4.7 
rejected observations, 3.18, 3.23 
rejection of observations, 3.23 
relative molar mass, 1.1 
reliability, consideration of, 5.23 
repeatability, 5.0, 5.2, 5.6 
reproducibility, 5.0, 5.2, 5.6 
residuals, 3.14, 3.19, 4.5 
resistance, acoustic, 10.1 
contact, 3.2 
measurement of, 3.2 
thermal, 10.1, 10.5 
resistance strain gauge bridge, 1.3, 1.4 
resolution of events, 3.26 
resolving power, 7.20 
resonant frequency, El. 5c, 7.20 
resonator, 6.7 

response, frequency, 7.12, 7.20 
Reynolds number, E1.5d 
Riabouchinsky, 1.3 
RFI, 5.7 
river flow, 5.13 
rms, definition of, 3.0 
rms deviation, 3.17, 3.19 
rms error, ideal, 3.17 

practical, 3.19 
See also error. 

rms error of the error, 3.22 

rms mean, 3.13 

roller bearings, 5.6 

root-mean-square. See rms. 

rounding, 3.25 

rss (root sum square), 5.19 

scales, 5.2 
sedimentation, 6.10 
segmental curve fit, 4.11, 4.18 
seismometer, 6.1 

selection, instrument, factors in, 3.35, 5.23 
self-heating, E3.11e 
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sensibility, definition of, 5.0 
ultimate, 5.19 
sensitivity, definition of, 5.0 

temperature coefficient of, 5.6 
sequence of measurements, 3.32 
serial combination of subsystems, 7.4 
series electrical circuit, 6.8 
series of systems, 10.5 
serviceability, consideration of, 5.23 
set of normal equations, 3.31, 4.10, 4.15 
set point, E3.11b, 5.1 
shielding, 5.7 
shock tube, 8. 1 
shot noise, 5.19 
signal-to-noise ratio, 5.20, 5.21 
significant digits, E3.22a, 4.12, 4.20 
Simpson’s rule, 5.13 
simultaneity, definition of, 6.0 
sinusoid, response to, 7.12, 7.20 
test by, 8.3 
SI units, 2.7 
skewed distribution, 3.16 
smoothing, 4.18 
S/N ratio, 5.20, 5.21 
solenoidal law, El.l 
sone, E2.8h 
sonic flow, E3.28/ 
sound, speed of, E2.11b 
units of, E2.8h 

sound pressure level, measurement of, 3.2 

spacing of observations, 4.8 

span, definition of, 3.4 

spatial average, 5.10 

spatial integration, 5.11, 5.13 

specification of accuracy, 3.34 

of an electrical instrument, E5.7a, 5.19 
of a pressure gauge, E5.7b,c 
specific gas constant, El.l 
spectrometer, 7.20 
spectrum analyzer, 8.3 
speed, molecular, E3.13d, E6.11b 
of light, 2.1, 3.1 
of sound, E2.11b 
spline, cubic, 4.18 
spring-mass system, 6. 1-6.3 
springs, natural frequency of, 8.4 

static characteristics of, 5.6 
square-law damping, 10.4 
square wave, averaging of, 5.16 
response to, 7.3 
standard auditor, E2.8h 
cell, E3.28J 

deviation. See rms error, 
observer, 1.2 
standards, 3.34, 5.22 


standards laboratory , 5.22 
stars, magnitudes of, E.2.8 /, E2.11a 
steady-state response, definition of, 7.5 
Stefan-Boltzmann constant, El.l 
Stefan’s law, El.l, 6.12 
step-change, response to, 7.7, 7.15 
test by, 8.1 
storage factor, 7.12 
STP, 3.1 

strain gauge, E5.17g 

resistance, 1.4 

strain measurement, errors in, 3.2 
sum of uniform distributions, 3.29 
synchronous detection or modulation, 5.21 
systematic errors, 3.10, 3.11, 3.28 

detection of existence of, 3.30 

t-distribution, 3.18 

temperature coefficient of sensitivity, 5.6 

measurement, errors in, 3.2, 5.8 
measurement, null method, 5.18 
tetrode noise, 5.19 
thermal capacitance, 10.1, 10.5 

conductivity of H 2 , E3.21e 

conductivity type of gas analyzer, E1.4f 

emfs, 5.7 

noise, 5.19 

resistance, 10.1, 10.5 

units, 2.3 

thermocouple, 3.2, E5.8a,b, E5.17c 
type E, E4.20a, 4.21 
W/Re, E4.16a 
thermometer, 6.12 

thermometer bulb, 3.2, E4.1a, 6.12, 10.1 
thermometry. See temperature measurement. 
Thevenin’s theorem, 5.18 
Thomas-type flowmeter, E5 . 1 1 e 
three-eighths rule, 5.13 
thrustmeter, 6.1 

time constant, definition of, 6.13 
of ratemeter, 3.27 
time lag, 7.7-7.21 
titration, electrometric, E3.28k 
torquemeter, 6.3 
transform, Laplace, 7.4 
transformed variables, 4.13, 4,14 
transient response, definition of, 7.5 
transistor noise , 5.19 
transmission system, electrical, 10.6 

pressure, 6.6, 10.5, 10.6 
trapezoidal rule, 5.13 

traverse, with probe, 5.12, 5.15, 5.17, 7.17, 7.18 
Trimmer, 7.4 
triode noise, 5.19 
truncated distribution, 3.18 
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truncation, 3.25 
tuning fork, El. 6b, E4.1a, 5.6 
turbine-type flowmeter, E5.17d 
turbulent profile , 5.10 

ultrasonic flowmeter, E5.11f 
uncertainty of correlation coefficient, 4.12 
of error, 3.22 

uniform error distribution, 3.18, 3.19 

distributions, sums of, 3.29 
units, list of equal, 2.1 
unit weight, 3.19 
unrationalized units, 2.5 

vacuum, E3.13d, 6.11 
vacuum measurement, errors in, 3.2 
vapor pressure, E2 . 1 1 c 
variance, 3.17 
velocimeter, 6.1, 6.3 
velocity of fluid in pipe, E5.15a,b 
vernier acuity, 5.19 
vibration absorber, dynamic, 9.3, 10.2 
measurement, 3.2 
vibrometer, 6.1, 6.3 


vigintile, 3.20 
viscometer, 6.10 
viscosity of N 2 , E4.22c 
of oil, 4.8 
visual acuity, 5.19 
visual magnitude, E2.8/ 
voltage measurement, null method, 1.4, 5.18 
voltmeter, E5.17b 
volume unit (VU), E2.8j 

weighted arithmetic mean, 3.13 
measurements , 3.19 

weighting, 3.21, 4.6, 4.13 
weights, calibration of, E3.31 
Weston cell, E3.28j 
Wheatstone bridge, El. 4c 
white noise, 5.19 
wire, hot. See hot wire, 
working band, 5.1 
Worthing, 3.14 

Zener-diode voltage reference, El. 4a 
zero-based straight line, 4.9 
zero, changes in, 5.6 



Index of Equations for Which Least-Squares Solutions are Given 


Equation 

Example 

II 

o 

* 

4.11a 

uj 

II 

4.13c 

j = a 0 + a t x 

4.11b 

y = a x x + 

4.11c 

-Cl 

3 

ii 

4.14.1a 

x = ay b 

4.14.1b 

y — aa bx 

4.14.1c 

x = aa by 

4. 14. Id 

y = 

4.14.1e 

x = aa b/y 

4.14. If 

y = a/(\ + bx) 

4. 14. lg 

X = a/( 1 + by) 

4.14.1h 

y = < 2 o + a x x + a^x 2 

4.1 Id 

y = a 0 + + tf2* 2 + a 3 * 3 (segment) 

4.1 le 

^ = ao + a x x + 

4.16b 

j = jc* + V 

4.16f 

y = aa^ +ci2 

4.14.2a 

y = <3« <* /jr> < c/jc2 > 

4.14.2b 

y — a / ( 1 + bx + ex 2 ) 

4.14.2c 

5 = + a 2 ^i + 03^2 + a 4^2 (T « Chebyshev polynomial) 4.16a 

5 = Oo + fljJC + <*2? + ^3^>’ 

4.16c 

j = Oo + a i* 4- a^x 2 + a$y + a 4 y 2 

4.16d 

5 = y*(a 0 + + ^3y + a*xy) 

4.16e 



Equation 


Section 

y = 

(k = 0,1,2,. . . 

, m) 

4.15 

y = Ea* •/*(*) 

CN 

It 

, m) 

4.16 

Y = y + r(dyldt) 



4.17 



NASA 

National Aeronautics and 
Space Administration 

Report Documentation Page 

1. Report No. 


2. Government Accession No. 3. Recipient’s Catalog No. 

NASA RP-1222 




4. Title and Subtitle 

Foundations of Measurement and Instrumentation 


5- Report Date 

April 1990 

6. Performing Organization Code 


7. Author(s) 

Isidore Warshawsky 


9. Performing Organization Name and Address 

National Aeronautics and Space Administration 
Lewis Research Center 
Cleveland, Ohio 44135-3191 

12. Sponsoring Agency Name and Address 

National Aeronautics and Space Administration 
Washington, D.C. 20546-0001 


8. Performing Organization Report No. 

E-3786 

10. Work Unit No. 

505-62-01 

1 1 . Contract or Grant No. 


13. Type of Report and Period Covered 
Reference Publication 

14. Sponsoring Agency Code 



16. Abstract 

This text is intended to provide to the user of instrumentation an understanding of the factors that influence 
instrument performance, selection, and application, and of the methods of interpreting and presenting the results 
of measurements. Such understanding is prerequisite to the successful attainment of the best compromise among 
reliability, accuracy, speed, cost, and importance of the measurement operation in achieving the ultimate goal of 
a project. Some subjects covered are dimensions; units; sources of measurement error; methods of describing and 
estimating accuracy; deduction and presentation of results through empirical equations, including the method of 
least squares; experimental and analytical methods of determining the static and dynamic behavior of 
instrumentation systems, including the use of analogs. 


17. Key Words (Suggested by Author(s)) 

Measurement; Instrumentation; Dimensions; Units; 
Accuracy of measurement; Curve fitting; Empirical 
equations; Errors of measurement; Dynamic response; 
Systems measurement; Instrument performance; 
Instrument selection 


18. Distribution Statement 

Unclassified — Unlimited 
Subject Category 35 


19. Security Classif. (of this report) 
Unclassified 


20. Security Classif. (of this page) 
Unclassified 


21 . No of pages 
240 


22. Price* 


NASA foam 1626 oct «6 * For by the National Technical Information Service, Springfield, Virginia 22161 


NASA- Langley, 1990 







